Yeah, that's an interesting challenge. One idea that makes sense to me is to split into different sub-agents when you feel like there's an in-principle tension or tradeoff between different kinds of values. For example, in a previous post I suggested different "buckets" for the broad goals of (i) pure suffering reduction, (ii) reliable global capacity growth, and (iii) high-impact long-shots. Lots of different concrete "causes" might compete within each philosophical bucket, but I wouldn't be tempted to break out a fourth bucket unless I felt like there w...
Say I compare different GHD interventions that help the worst off. I know that most such interventions are not crazy effective and that it's easy to under or overestimate those with little evidence. I have a good reference class. If someone tells me about a new intervention in the area, I don't expect it to be crazy good. It's much more likely to be close to the mean. I have a prior expectation. If my math tells me there's some poorly-studied intervention that beats the one that has consistently proven most effective so far, this weak evidence should not o...
Hi Dan/Fable, thanks for the critique! The three most important problems I see:
1. Claiming that the unawareness argument only has practical implications if there's a privileged “default”
The sequence argues carefully for incomparability. It cannot argue that incomparability favors inaction, because if A and the status quo are incomparable, the status quo is not better. Yet the practical gloss everyone puts on the conclusion resolves every incomparability toward the default
As discussed here and in the introduction of the sequence, my claim was never “we shou...
This has been helpful for me. Thank you! I've had some inner unease about trade-offs I made between various career choices and in other morally relevant projects. Your post gives me a more principled way to think about these choices.
A practical question: Do you have some advice on the breadth and depth of inner sub-agents? There are many causes and worldviews that I could learn about, particularly being someone who easily gets distracted and hooked on new ideas. I can't give adequate consideration to all of them, unless I group them into a manageable numbe...
The NYC one will have more people, so I think you'd find more people to 1-1 on US-China AI governance. But it depends on how many 1-1s you want to have. If it's just a few, then both are equal; if you are willing to have 10+ 1-1s on US-China AI governance, I imagine it'd be slightly harder to get that at Berkeley.
To clarify, centering people's preferences here refer to centering their telic desires / fundamental moral weights, the implications of which (when used in a CEA to assess various individual actions/philanthropic interventions/government policies) may imply that people's actions are irrational or non-maximizing.
Hi Kristof, if the steelman case is that autonomy is valuable and we should be averse to using coercion unless there is strong justification, then I do agree. However, I would add that I - and many others - see public health (i.e. saving lives and preventing unnecessarily disability/pain) as pretty good justification. Also, in this case, we're talking about taxation and regulatory interventions; people aren't banned from e.g. eating salty good or drinking sugary drinks. A key point too is that because our tastes are adaptive, people don't even notice reduc...
As someone who spent six months in Oxford and four years in London and has now moved back home to Melbourne, for me I suspect there has been a large drop in impact. Even though I go to conferences once or twice a year, I think the difference in quality of networking, deeper connections, more fruitful conversations at the watercooler, etc is quite dramatic.
Also being outside of a hub but in the same timezone and same country as a lot of EA employers is less of a hit to impact than living in a different country and a very different time zone, where it's harder to schedule meetings and there are far fewer jobs that would be willing to hire.
I think partly this is a reaction to dishonest charity fundraising communications that say things like “Donate today and your gift will be matched 8×”, when in reality either the match limit is virtually guaranteed to met or else the matching donor would offset any unused matching funds with a separate donation.
So consider your last sentence — maybe if I do the thing you will give an extra $100 at the end of the year or if I don’t do the thing you will give an extra $150 at the end of the year. Technically I only gave the $50 gift if you did the thing, but in reality there was no counterfactual difference in total giving.
You're splitting this the same way I meant to, though the quote you point out wasn't as clear as it could have been: in strictly verifiable domains, AIs will eventually be so much better than humans that having humans involved will only lead to worse decisions. But the line between verifiable and unverifiable is gradual and fuzzy. My expectation in practice is that humans will develop such a strong habit of deferring to AIs that even if there are still humans "in charge" of certain things in the sense that they could implement one decision or another, they...
Has the forum team considered making a settings option to hide vote counts? The option to hide agree/disagree and reaction counts are something I would also appreciate.
I stayed away from this forum for over a year because of it being structured like Reddit, and have also heard concerns about vote counts anywhere influencing bias, as well as issues with like and dislike buttons in general. This post on LessWrong makes a more ambitious case for this, though I'm not campaigning here for any change affecting all users.
I understand the voting system is a necess...
Hi all, introducing myself here: I'm Eleri (or Elli) Thomas.
I crossed paths with early EA and 80,000 Hours while at Oxford (graduated 2012), though I haven't been active in the community since. Since then I've had a rather varied, interdisciplinary career as an urbanist (trained in history, urban design and town planning, and worked across the built environment professions, economics, infrastructure, smart cities, and policy, design, development and strategy). I've been at think tanks (Centre for Cities), advised government/local authorities/developers (De...
Hey David! This is great, thanks for sharing, I'll take a more detailed look. For now, some comments:
I believe the best way is to call them out on it. If the number of these failures keeps rising like that, there's a risk of it becoming a regular occurrence. Whenever you see it, point it out by commenting on the post itself or share it with your thoughts, or write in your blog like I did. That will make them double and maybe triple check next time.
@robirahman this feature is now live - you can submit multiple applications from the same account, just go back to https://app.grantmaking.ai/apply/launch and it will say "Start another application"
You can only have one saved draft at a time though. So you have to submit application 2 before you start application 3
A counterfactual donation isn't just my act being contingent on your act, it means AMF getting a net extra $50 is contingent on your act. So if part of the $50 were a donation match that would be filled regardless, or some (especially low op cost) funder would make up the difference, it wouldn't fully count.
Counterfactual is also used to mean the second choice opportunity that defines an opportunity cost. "Should I take job A? Well my counterfactual is job B, where I would do X..."
Ah, thanks for sharing that! I really loved that article!
I feel like this explains a lot of my interactions with people, where they'll reject a crazy idea I have (such as that humanity should eventually work to eliminate wild animal suffering), but they won't really offer a coherent argument against it. It's as though they have an immune system for crazy beliefs that activates above a certain threshold.
Thanks for commenting!
In my mind, you should just ask for the min funding it would take for you to quit your job. So if you have a research path or project idea, and you need $30k for 6 months to quit your job and make it happen, make that your application and funding ask.
Because if you set your min funding to $30k you will either not get funded or you get the $30k and can quit your job.
Does that make sense? Or did you have something else in mind?
Hi Joel, thank you for the detailed and thoughtful comment!
(1) I agree that the post mischaracterized the report. My apologies for that. I've updated the corresponding section to make it clear that the adjustments were made due to the surveyed group being unrepresentative of the intervention's target, not because of circular reasoning.
(2) I agree that autonomy shouldn't be privileged a priori above life, health, or any other values, but this is the more committed harm reductionist view the post is describing, not defending. What I am arguing for is somethi...
Great ideas and important concepts that needs to be discussed more widely. Although I don't find the base dichotomy of Ordinary, and Extra-ordinary to be very reliable or natural analogy with Bonobos, and Chimps.
I think the fundamental difference between them is meaningful and significant enough agency. The "ordinary" is way more extra than "extra-ordinary" the moment they develop awareness and realization to exercise their agency or something great.
What you describes sounds to me like the regular grant request in the future, where the minimum amount is not $5k, but some larger amount that would let you quit your job .
I encourage you to apply when you feel ready!
There isn't an explicit coordination mechanism at the moment, but my impression is that if you had a good application: there would be someone willing to fund the whole amount. Currently there seems to be access of funding, and not enough great applicants
Hey Dan! In this case we have decided to leave this up and count you as eligible (to be clear, for the first stage, it could still be a desk rejection in the end) because this already has some valuable discussion. From now on we'll ask people to take their posts down if they are posting and hoping to be included in the competition. I'll also make sure that the announcement post is clearer. Cheers!
This is an exciting opportunity. I'm especially glad to see more emphasis on governance and practical responses alongside foundational research. Even if there's uncertainty about digital minds today, thinking through institutional, legal, and ethical frameworks in advance seems like a valuable investment. Looking forward to seeing the range of proposals this RFP attracts.
I authored CEARCH's deep report on hypertension, which informs both our own grantmaking in GHD and Charity Entrepreneurship's recommendation of salt reduction policy as a promising idea to be incubated. CEARCH's report was cited by the OP Kristof in his post. Kristof says "The survey was administered to EAs, who were then noted to be disproportionately libertarian due to their high education levels, and their autonomy concerns are discounted on those grounds. The framework ends up treating the strength of someone's concern about autonomy as a reason to giv...
Quick follow-up: I've worked with Claude Code (several iterations; wanted Fable, got blocked so pushed to Opus) to put up this page comparing and bridging Pablo's model to ours...
This is early -- I continue vetting and working on it and would love your feedback
There's a potential complementarity: we focus on is the cost distribution, Pablo on the demand response is Pablo's
(If you add hypothes.is comments there I'll respond and adapt.)
Thanks for this! We've been doing some related work through The Unjournal Pivotal questions -- see this modeling interface/calculator. I'm reading through your modeling and discussion carefully now, aiming to do some comparisons between our approaches, and it might be worth some conversation/collaboration (feel free to DM).
Also see the resources and evidence coming out of our recent workshop, which we're following up on, aiming to get a wider range of views and a productive crux-mappiung conversation between 'skeptics and optimists'. We plan to share fur...
Hi Simon. In that case, how do you explain that impact-focussed grantmakers support many interventions, even within a single area (see, for example, the Animal Welfare Fund)? If the cost-effectiveness of each did not meaningfully decrease with spending, one would expect them to focus on much fewer grants?
You might also be interested in this old post, tangentially related to this topic https://slatestarcodex.com/2019/06/03/repost-epistemic-learned-helplessness/
could you link the blog post as a Google docs here? (could be with access on request if you're feeling shy).
I might do a first pass of the curriculum on my spare time and I'd love to piggyback on someone else's previous thinking.
I'd recommend you spend absolutely zero time polishing it before putting it here if you think that will make it less like that you share it (see my blog for a somewhat brazen embrace of that principle: https://myea.blog)
Seems like a decent intro to the topic, but it's not what I'm interested on.
On a quick skim, they seem to concern themselves with "baseline" virtues, a standard that most utilitarians should follow, derived from simple, common-sense considerations. I'm much more interested on "speculative" or "frontier" virtues. Character traits that, if relentlessly followed, could expose you to a much higher impact ceiling. Some candidates:
Shoutouts to the EA Impact Hubs!
"Impact Hubs are affordable, temporary coworking and coliving setups for EAs and impact-focused people. The idea is to make it easier to do focused work and build connections with other EAs without having to be in an expensive EA hub like London or SF."
Also I’m not sure if anyone specifically in the EA space is working on it yet, but there is a case for government e-procurement development in low & middle income countries saving a lot of money & reducing corruption.
https://www.cambridge.org/core/journals/journal-of-benefit-cost-analysis/article/investment-case-for-egovernment-procurement-a-costbenefit-analysis/1BC5D35109D488269F4C8F3E95C0B714
But it is still dependent on like a Minister of Finance & others willing to implement it & reduce the corruption in their domain.
I used to think the same, but I recently changed my mind, for reasons similar to Zach's
Some quick thoughts:
Significant additional spare money
I agree money is great, but I think moving to a HCOL area will often increase the amount of money you can earn/save for most people, by giving you higher-earning jobs opportunties. Even if you're doing self-funded research, the amount of money you can influence will likely increase enough by moving to a HCOL area to compensate your higher costs. It's much easier to help people give more and give better if they're weal...
I'm very far from working in the AI industry, so I don't think I'm the target audience of this post.
In any case, as someone trying to use part of my resources to help others, I agree with much of it (especially points 1. 2. and 6.) but I personally strongly disagree with other points:
3. The first priority in charity is to mitigate potential harms caused by how you made your
money.
For people working in AI the potential harms of their work are immense, and involve most present and future life, so I think it's reasonable to prioritize mitigating those harms, ...
On your first sentence, not sure if you read footnote 3 - I believe I cover under the "value of the future" bullet point why I didn't include this. (combined with AIs probably being moral patients).
And your three bullet points, very briefly:
“Maximising expected choiceworthiness involves diversification because the cost-effectiveness of each intervention decreases with spending. Going "all in" on the option with the highest marginal cost-effectiveness would only make sense for a very limited budget”
I’d argue almost all decision makers in the EA space operate in the linear regime where cost effectiveness does not meaningfully decrease.
Surely the most effective way of making change is to influence and in fact directly infiltrate the means of change i.e. through policy and policymaking.
I've been working across research/policy and practice for the last 12 years, and yes it's incredibly difficult to build relationships, trust and traction but I think that's more of a reason to actively engage with it rather than to turn away. I've been particularly interested in the energy in the UK from the "YIMBY" movement, Looking for Growth are a good example of this, and the attitude of "you can just ...
The part I'm most curious about here is where you say that "keeping humans in the loop will only degrade decision-making."
I'm fairly new to the AI alignment debates, so I'm not sure how widely accepted this assumption is (and I'd love to be challenged on this) but if it is, I find it quite concerning;
It seems reasonable to design systems that can help humans make better-informed decisions, and to act autonomously where that's appropriate, but that's quite different from concluding that humans should be removed from morally significant decision-making becau...
Future life may turn out to be net negative (e.g. s-risks might become real), and work that preserves the chances of humanity spreading may essentially be enabling that.
In almost the opposite direction, AI might just be very useful and beneficial and a bunch of the risks people in AI safety worry about might not end up becoming real, or might be quite easy to overcome. And in the meantime, AI safety work may have:
Thanks for this! Firstly these seem like two separate Qs (“which actors am I most worried about?” vs the specific risks posed by defender-first access programmes).
My current guess (as a relatively uninformed outsider) is that the most worrying near-term category actors with some combination of pre-existing domain knowledge, operational access, and (obviously) intent. So in bio I guess this would mean: state/state-adjacent programmes, small highly motivated teams with wet-lab access, and maybe compromised actors inside institutions that give them access. I ...
Hi, thanks for this post and for your engagement :) If you haven't checked them out yet there are quite a lot of resources on similar topics, which may not answer all the questions you have but can be a good starting point (and can also help put you in touch with others who are dealing with similar things).
See for example this post from 2022, the comments and the resources it links to (there are probably more recent ones too):
https://forum.effectivealtruism.org/posts/Hyco4iMbL6phJwCH2/ea-career-guide-for-people-from-lmics
Hi Renee,
Thank you for your thoughtful comment. I have been thinking about this ever since, and it is interesting that things are moving forward.
Since my post, I talked to High Impact Professionals, and they are indeed adding a marker on people's profiles when they are top candidates. That is a good outcome.
I do not know if I find it interesting to have a work test straight away. The thing about putting the work test straight away in the application is that it would make it so that fewer people would apply, or you would have a lot of low-quality tests. I w...
Hey, Moneer, great to hear your views on this
To be thorough, I did point out the mistake in the Africa image to the page on LinkedIn that published it, and asked them to tell me which AI they'd used and the prompt. They didn't respond.
My CELTA tutor told me once that many trainees would submit written assignments with grammatical and structural errors, and these are always underlined in the word doc in red, no less.
It would be great to really know why someone skips checking AI outputs, given that it's prone to hallucinations. They might be the same ones who submitted to my tutor :)
Until then, we just need to be watchful and call out the mistakes. Maybe someone will pay more attention because of it.
I remain sceptical of computational functionalism (CF). It implies some sets of AND, OR, and NOT operations lead to consciousness even if run at the rate of one operation every billion years, which seems very implausible to me.
Hi Kariema, I appreciated the visual examples and hyperlinked cases of "human failure". Stories to share around a dinner table!
I agree in principal that individuals have a responsibility to maintain oversight. However, I think the characterisation of "human failure" as an individual's problem (i.e. someone "not bothering to check") and the proposed solution of "us standing guard" is only a piece of a wider puzzle that necessitates identifying and addressing the complexity of situational/ structural factors (see fundamental attribution error).
In each of t...
Agree, I think people should "be the change they want to see" more often, especially in community / field building.
One caveat: Understand reputational harm risks and manage them (don't overestimate them either)
If your projects includes using the EA brand for outreach or other public comms, I'd recommend at least informing CEA or a local (full-time, paid) EA community builder, and asking if there's anything you should consider.
In some cases, there might be downside risks to other EA orgs or EA as a whole, some of which you might overlook, especially if y...
Thanks for this Michelle (and Pause AI).
Being located in Brussels, it always feels that there must be ways to influence the people within the EU who have so much power. Not just the politicians (for whom there are thousands of professional and well-paid full-time lobbyists), but also the eurocrats who manage a lot of the tangible work the EU does.
It always feels like the bad guys (the oil-companies and tobacco companies and people looking for reduced regulation (often using the euphemism "simplification" to describe what is really reducing consumer-protec...
Writing in my personal capacity. Abhi and I have also discussed the possibility of working together on this topic.
I appreciate your engaging with my review of alt proteins. I agree that taste is somewhat underspecified there. One challenge I ran into is that proponents of improving taste don't often specify exactly how they think taste equivalence should be operationalized. I tried to cover the range of operationalizations for which I could find evidence, but ended up focusing primarily on human taste tests rather than analytical chemical evidence. My reas...
If you or someone you know needs funding to get their AI safety research / career / project / think of the ground, apply for funding through the $1M AI x-risk grant round we just launched on grantmaking.ai
Apply by July 13 to guarantee a review: https://grantmaking.ai/apply/launch
Thanks for sharing your perspective! The unpaid-signalling bottleneck you describe for generalists resonates a lot.
One idea I'm exploring is a lightweight service that connects skilled freelancers to AI safety orgs
If you want to launch a pilot (e.g., paid trial projects, standardized vetting, or a small marketplace/concierge), we're running a $1M Launch Round for small, early-stage AI-safety bets on grantmaking.ai - I encourage to you apply.
July 13 is the deadline to guarantee a review: https://grantmaking.ai/apply/launch
Hi Vasco, thanks for the initiative! Hopefully we can get some locals to partake in the hub, and I assume there'll be engagement with the Lisbon crowd too.
We deliberately decided not to put deadlines on the applications since we don't want to turn interested people away only because they found out too late. And the application forms have been combined since shortly after we published this post, perhaps you were looking at an old version?
You're absolutely right to bring this to attention. You mention many examples of this specific kind of human failure that's becoming more and more common every day and yet they are just a fraction, the real numbers are way higher and we do not have to look very far to find them. This hurts in so may ways.
Thanks for this comment, William. I think that your third point is especially strong.
While I'm not worried about the 140 million potential consumers for the reasons mentioned, and while I don't think these bans indicate broad meat industry antipathy toward alt proteins (as discussed in the book), I have become convinced that I did not give enough weight to the potential chilling of public and private investment.
Often these bans are cited for the idea that the meat industry is mobilizing against alt proteins, but most of the meat industry opposes these ba...
GiveWell has lots of research on evaluating programs. Check out their website and podcast, and their grant recipients.
Happier Lives Institute does a similar thing to GiveWell, but with a different moral prioritisation (and a substantially smaller budget).
Giving Green and Power for Democracies do similarly, and I think Peace Per Dollar might be starting up.
Giving What We Can has its "other supported charities" list.
Ambitious Impact has lists of EA-incubated charities (but again, this is a different thing from effective giving opportunities, as they're often...
What are the central virtues a committed utilitarian should cultivate? I would kill for a really great answer to this question
I think Stefan Schubert & Lucius Caviola's Virtues for Real-World Utilitarians is pretty good!
Assuming we end factory farming before 2100, which factor will have contributed the most?
I don't have a lot of hope for cultivated meat. But I have more for the plant based meat we already have, and I think it's already instrumental in getting more people to go vegan. That said, I don't think any meat alternatives will get us all the way (I think so far we mostly have evidence against that), so social change seems essential.
It's great that you're thinking about this!
I'm confused why you are denominating options in robotics-startup-days saved. This feels like a narrow definition of "impact". I'd encourage you to consider other ways to benefit the world; parts 4-6 of the 80k career guide might be helpful. Specifically, under the assumption that the thing you terminally value is more like "reducing suffering" than "robotics progress", I would encourage you to first consider which causes advance those values, and only then drill into job options. (The 80k career guide will walk ...
Thank you for this and hope this will generate more discussion.
Have you considered the role technology, especially AI can play in this? Can some of what regrantors do be replaced or supplemented? AI is capable of processing large amounts of data and doing it on a ongoing basis, something that can be very, if not prohibitively costly for current donors and regrantors, especially at smaller scale. It can help achieve much more granular allocation: smaller projects and initiatives, individual needs, continuous rather than cyclical donations.
Shortening the dis...
I think this is an important question more people should be thinking about (particularly if one believes that there is going to be a significant increase in philanthropic funding due to AI money).
Rather than refer to research I can write about my personal reasons/takes:
I primarily work in global health (and AI safety), and my main reason for doing so is:
measurability: it's possible to measure the impact of global health projects fairly rigorously, compared to democracy promotion or economic growth. Additionally, many global health problems are probably jus...
Happier lives Institute: https://www.happierlivesinstitute.org/
EA also does democracy charity evaluation which you might want to look into https://www.powerfordemocracies.org/
Hi James! There's actually a variety of EA-aligned charities in those areas. Some that come to mind:
They all have their own theories as to why they might be unusually effective.
As for why:
Hey Dan, just to clarify for you and other readers, this post wouldn't be eligible for the competition, because a) we aren't accepting pieces that have already been published and b) every entry must be submitted via the entry form.
Sorry if this wasn't clear in the announcement post! I'll edit it next week to make this plainer.
Also - to add, if you're UK based, not sure if Impact Ops would consider taking on freelancers to support their consulting offerings to clients, but that could be another avenue.
Cool. Thanks for sharing!