Denis Drescher

I’m mostly interested in global priorities research, the questions listed in the research agendas of CLR and GPI, and computational methods for approaching them. More on that in my post on my self-study directions.

I currently dedicate 60–80% of my time to this self-study and may occasionally write about my latest insights.

If you’re also interested in less directly optimific things – such as climbing around and on top of boulders or amateurish musings on psychology – then you may enjoy some of the posts I don’t cross-post from my blog, Impartial Priorities.

(Pronouns “they” or “he.” But I’m also happy with a bunch of other pronouns.)

Comments

Interrogating Impact

Re 3: Yes and no. ^.^ I’m currently working on something of whose robustness I have very weak evidence. I made a note to think about it, interview some people, and maybe write a post to ask for further input, but then I started working on it before I did any of these things. It’s like an optimal stopping problem. I’ll need to remedy that before my sunk cost starts to bias me too much… I suppose I’m not the only one in this situation. But then again I have friends who’ve thought for many years mostly just about the robustness of various approaches to their problem.

Hilary Graves doesn’t seem to be so sure that robustness gets us very far, but the example she gives is unlike the situations that I usually find myself in.

Arden Koehler: Do you think that’s an appropriate reaction to these cluelessness worries or does that seem like a misguided reaction?

Hilary Greaves: Yeah, I don’t know. It’s definitely an interesting reaction. I mean, it feels like this is going to be another case where the discussion is going to go something like, “Well, I’ve got one intervention that might be really, really, really good, but there’s an awful lot of uncertainty about it. It might just not work out at all. I’ve got another thing that’s more robustly good, and now how do we trade off the maybe smaller probability or very speculative possibility of a really good thing against a more robustly good thing that’s a bit more modest?”

Hilary Greaves: And then this feels like a conversation we’ve had many times over; is what we’re doing just something structurally, like expected utility theory, where it just depends on the numbers, or is there some more principled reason for discarding the extremely speculative things?

Arden Koehler: And you don’t think cluelessness adds anything to that conversation or pushes in favor of the less speculative thing?

Hilary Greaves: I think it might do. So again, it’s really unclear how to model cluelessness, and its plausible different models of it would say really different things about this kind of issue. So it feels to me just like a case where I would need to do a lot more thinking and modeling, and I wouldn’t be able to predict in advance how it’s all going to pan out. But I do think it’s a bit tempting to say too quickly, “Oh yeah, obviously cluelessness is going to favor more robust things.” I find it very non-obvious. Plausible, but very non-obvious.

She has thought about this a lot more than I have, so my objection probably doesn’t make sense, but the situation I find myself in is usually different from the one she describes in two ways: (1) There is no one really good but not robust intervention but rather everything is super murky (even whether the interventions have EV) and I can usually think of a dozen ways any particular intervention can backfire; and (2) this backfiring doesn’t mean that we have no impact but that we have enormous negative impact. In the midst of this murkiness, the very few interventions that seem much less murky than others – like priorities research or encouraging moral cooperation – stand out quite noticeably.

Re 4: I’ve so far only seen Shapley values as a way of attributing impact, something that seems relevant for impact certificates, thanking the right people, and noticing some relevant differences between situations, but by and large only for niche applications and none that are relevant for me at the moment. Nuno might disagree with that.

I usually ask myself not what impact I would have by doing something but which of my available actions will determine the world history with the maximal value. So I don’t break this down to my person at all. Doing so seems to me like a lot of wasted overhead. (And I don’t currently understand how to apply Shapley values to infinite sets of cooperators, and I don’t quite know who I am given that there are many people who are like me to various degrees.) But maybe using Shapley values or some other, similar algorithm would just make that reasoning a lot more principled and reliable. It’s well possible.

Interrogating Impact

Sorry if you’re well aware of these, but points 3 and 4 sound like the following topics may be interesting for you: For 3, cluelessness and the recent 80k interview with Hilary Greaves that touches on the topic. For 4, Shapley values or cooperative game theory in general. You can find more discussions of it on the EA Forum (e.g., by Nuno), and I also have a post on it, but it’s a couple years old, so I don’t know anymore if it’s worth your time to read. ^.^'

Are there any other pro athlete aspiring EAs?

I’d like to keep up-to-date on what you’re doing. I don’t have chance getting anywhere close to an interesting level anymore in the sport that I do (climbing, mostly bouldering), but I might occasionally meet those who do. (No worries, I can be tactful. ^^)

AMA: Owen Cotton-Barratt, RSP Director

I’ve thought a bit about this for personal reasons, and I found Scott Alexander’s take on it to be enlightening.

I see a tension between the following two arguments that I find plausible:

  1. Some people run into health issues due to a vegan diet despite correct supplementation. In most cases it’s probably because of incorrect or absent supplementation, but probably not in all. This could mean that a highly productive EA doing highly important work may cease to be as productive with a small probability. Since they’ve probably been doing extremely valuable work, this decrease in output may be worse than the suffering they would’ve inflicted if they had [eaten some beef and had some milk](https://impartial-priorities.org/direct-suffering-caused-by-various-animal-foods.html). So they should at least eat a bit of beef and drink a bit of milk to reduce that risk. (These foods may increase other risks – but let’s assume for the moment that the person can make that tradeoff correctly for themselves.)
  2. There is currently in our society a strong moral norm against stealing. We want to live in a society that has a strong norm against stealing. So whenever we steal – be it to donate the money to a place where it has much greater marginal utility than with its owner – we erode, in expectation, the norm against stealing a bit. People have to invest more into locks, safes, guards, and fences. People can’t just offer couchsurfing anymore. This increase in anomie (roughly, lack of trust and cohesion) may be small in expectation but has a vast expected societal effect. Hence we should be very careful about eroding valuable societal norms, and, conversely, we should also take care to foster new valuable societal norms or at least not stand in the way of them emerging.

I see a bit of a Laffer curve here (like an upside-down U) where upholding societal rules that are completely unheard of has little effect, and violating societal rules that are extremely well established has little effect again (except that you go to prison). The middle section is much more interesting, and this is where I generally advise to tread softly. (But I’m also against stealing.)

Because the way I resolve this tension for me is to assess whether in my immediate environment – the people who are most likely to be directly influenced by me – a norm is potentially about to emerge. If that is the case, and I approve of the norm, I try to always uphold that norm to at least an above average level.

                                                                                           

Well, and then there are a few more random caveats:

  1. As the norm not to harm other animals for food becomes stronger, it’ll be less socially awkward for people (outside vegan circles) to eat vegan food. Social effects were (last time I checked) still the second most common reason for vegan recidivism.
  2. As the norm not to harm other animals for food becomes stronger, more effort will be put into providing properly fortified food to make supplementation automatic.
  3. Eroding a budding social norm because it comes at a cost to one’s own goals seems like the sort of freeriding that I think the EA community needs to be very careful about. In some cases the conflict is only due to lacking idealization of preferences or only between instrumental rather than terminal goals or the others would defect against us in any case, but we don’t know any of this to be the case here. The first comes down to unanswered questions of population ethics, the second to the exact tradeoffs between animal suffering and health risks for a particular person, and the third to how likely animal rights activists are to badmouth AI safety, priorities research, etc. – probably rarely.
  4. Being vegan among EAs, young, educated people, and other disproportionately antispeciesist groups may be more important than being vegan in a community of hunters.
  5. A possible, unusual conclusion to draw from this is to be “private carnivor”: You only eat vegan food in public, and when people ask you whether you’re vegan, you tell them that you think eating meat is morally bad, a bad norm, and shameful, and so you only do it in private and as rarely as possible. No lies or pretense.
  6. There’s also the option of moral offsetting, which I find very appealing (despite these criticisms – I think I somewhat disagree with my five-year-old comment there now), but it doesn’t seem to quite address the core issue here. 
  7. Another argument you mentioned to me at an EAGx was something along the lines that it’ll be harder to attract top talent in field X (say, AI safety) if they not only have to subscribe X being super important but have to subscribe to  X being super important and be vegan. Friend of mine solve that by keeping those things separate. Yes, the catering may be vegan, but otherwise nothing indicates that there’s any need for them to be vegan themselves. (That conversation can happen, if at all, in a personal context separate of any ties to field X.)
The Case for Education

Interesting, thank you! Assuming there are enough people who can do the “normal good things EAs would also do,” that leaves the problem that it’ll be expensive for enough people with the necessary difference in subject-matter expertise to devote time to tutoring.

I’m imagining a hierarchical system where the absolute experts on some topic (such as agent foundations or s-risks) set some time aside to tutor carefully junior researchers at their institute; those junior researchers tutor somewhat carefully selected amateur enthusiasts; and the the amateur enthusiasts tutor people who’ve signed up for (self-selected into) a local reading club on the topic. These tutors may need to be paid for this work to be able to invest the necessary time.

This is difficult if the field of research is new because then (1) there may be only a small number of experts with very little time to spare and no one else who comes close in expertise or (2) there may be not yet enough knowledge in the area to sustain three layers of tutors while still having a difference in expertise that allows for this mode of tutoring socially.

But whenever problem 2 occurs, the hierarchical scheme is just unnecessary. So only problem 1 in isolation remains unsolved.

Do you think that could work? Maybe this is something that’d be interesting for charity entrepreneurs to solve. :-)

What would also be interesting: (1) How much time do these tutors devote to each student per week? (2) Does one have to have exceptional didactic skills to become tutor or are these people only selected for their subject-matter expertise? (3) Was this particular tutor exceptional or are they all so good?

Maybe my whole idea is unrealistic because too few people could combine subject-matter expertise with didactic skill. Especially the skill of understanding a different, incomplete or inconsistent world model and then providing just the information that the person needs to improve it seems unusual.

The Case for Education

Hi Chi! I keep thinking about this:

My tutor pushed back and improved my thinking a lot and in a way that I frankly don't expect most of the people in my EA circle to do. I hope this also helps me evaluate the quality of discussion and arguments in EA a bit although I'm not sure if that's a real effect.

If you have a moment, I’d be very interested to understand what exactly this tutor did right and how. Maybe others (like me) can emulate what they did! :-D

Objections to Value-Alignment between Effective Altruists

I’ve come to think that evidential cooperation in large worlds and, in different ways, preference utilitarianism pushes even antirealists toward relatively specific moral compromises that require an impartial empirical investigation to determine. (That may not apply to various antirealists that have rather easy-to-realize moral goals or one’s that others can’t help a lot with. Say, protecting your child from some dangers or being very happy. But it does to my drive to reduce suffering.)

Objections to Value-Alignment between Effective Altruists

Thank you for writing this article! It’s interesting and important. My thoughts on the issue:

Long Reflection

I see a general tension between achieving existential security and putting sentient life on the best or an acceptable trajectory before we cease to be able to cooperate causally very well anymore because of long delays in communication.

A focus on achieving existential security pushes toward investing less time into getting all basic assumptions just right because all these investigations trade off against a terrible risk. I’ve read somewhere that homogeneity is good for early-stage startups because their main risk is in being not fast enough and not in getting something wrong. So people who are mainly concerned with existential risk may accept being very wrong about a lot of things so long as they still achieve existential security in time. I might call this “emergency mindset.”

Personally – I’m worried I’m likely biased here – I would rather like to precipitate the Long Reflection to avoid getting some things terribly wrong in the futures where we achieve existential security even if these investigations comes at some risk of diverting resources from reducing existential risk. I might call this “reflection mindset.”

There is probably some impartially optimal trade off here (plus comparative advantages of different people), and that trade off would also imply how much resources it is best to invest into avoiding homogeneity.

I’ve also commented on this on a recent blog article where I mention more caveats.

Ideas for Solutions

I’ve seen a bit of a shift toward reflection over emergency mindset at least since 2019 and more gradually since 2015. So if it turns out that we’re right and EA should err more in the direction of reflection, then a few things may aid that development.

Time

I’ve found that I need to rely a lot on others’ judgments on issues when I don’t have much time. But now that I have more time, I can investigate a lot of interesting questions myself and so need to rely less on the people I perceive as experts. Moreover, I’m less afraid to question expert opinions when I know something beyond the Cliff’s Notes about a topic, because I’ll be less likely to come off as arrogantly stupid.

So maybe it would help if people who are involved in EA in nonresearch positions were generally encouraged, incentivized, and allowed to take off more time to also learn things for themselves.

Money

The EA Funds could explicitly incentivize the above efforts but they could also explicitly incentivize broad literature research and summarization of topics and interviews with topic experts for topics that relate to foundational assumptions in EA projects.

“Growth and the Case Against Randomista Development” seems like a particular impressive example of such an investigation.

Academic Research

I’ve actually seen a shift toward academic research over the past 3–4 years. And that seems valuable to continue (though my above reservations about my personal bias in the issue may apply). It is likely slower and maybe less focused. But academic environments are intellectually very different from EA, and professors in some field are very widely read in that field. So being in that environment and becoming a person that widely read people are happy to collaborate with should be very helpful in avoiding the particular homogeneities that the EA community comes with. (They’ll have homogeneities of their own of course.)

Denis Drescher's Shortform

Studies on Slack” by Scott Alexander: Personal takeaways

There have been studies on how software teams use Slack. Scott Alexander’s article “Studies on Slack” is not about that. Rather it describes the world as a garlic-like nesting of abstraction layers on which there are different degrees of competition vs. cooperation between actors; how they emerged (in some cases); and what their benefit is.

The idea, put simply, at least in my mind, is that in a fierce competition innovations need to prove beneficial immediately in logical time or the innovator will be outcompeted. But limiting innovations to only those that either consist of only one step or whose every step is individually beneficial is, well, limiting. The result are innovators stuck in local optima unable to reach more global optima.

Enter slack. Somehow you create a higher-order mechanism that alleviates the competition a bit. The result is that now innovators have the slack to try a lot of multi-step innovations despite any neutral or detrimental intermediate steps. The mechanisms are different ones in different areas. Scott describes mechanisms from human biology, society, ecology, business management, fictional history, etc. Hence the garlic-like nesting: It seems to me that these systems are nested within each other, and while Scott only ever describes two levels at a time, it’s clear enough that higher levels such as business management depend on lower levels such as those that enable human bodies to function.

This essay made a lot of things clearer to me that I had half intuited but never quite understood. In particular it made me update downward a bit on how much I expect AGI to outperform humans. One of mine reasons for thinking that human intelligence is vastly inferior to a theoretical optimum is that I thought evolution could almost only ever improve one step at a time – that it would take an extremely long time for a multi-step mutation with detrimental intermediate steps to happen through sheer luck. Since slack seems to be built into biological evolution to some extent, maybe it is not as inferior as I thought to “intelligent design” like we’re attempting it now.

It would also be interesting to think about how slack affects zero-sum board games – simulations of fierce competition. In the only board game I know, Othello, you can thwart any plans the opponent might have with your next move in, like, 90+% of cases. Hence, I made a (small but noticeable) leap forward in my performance when I switched from analyzing my position through the lens of “What is a nice move I can play?” to “What is a nice move my opponent could now play if it were their turn and how can I prevent it?” A lot of perfect moves, especially early in the game, switch from looking surprising and grotesk to looking good once I viewed them through that lens. So it seems that in Othello there is rarely any Slack. (I’m not saying that you don’t plan multi-step strategies in Othello, but it’s rare that you can plan them such that actually get to carry them out. Robust strategies play a much greater role in my experience. Then again this may be different at higher levels of gameplay than mine.)

Perhaps that’s related to why I’ve seen not particularly smart people yet turning out to be shockingly efficient social manipulators, and why these people are usually found in low-slack fields. If your situation is so competitive that your opponent can never plan more than one step ahead anyway, you only need to do the equivalent of thinking “What is a nice move my opponent could now play if it were their turn and how can I prevent it?” to beat, like, 80% of them. No need for baroque and brittle stratagems like in Skyfall.

I wonder if Go is different? The board is so big that I’d expect there to be room to do whatever for a few moves from time to time? Very vague surface-level heuristic idea! I have no idea of Go strategy.

I’m a bit surprised that Scott didn’t draw parallels to his interest in cost disease, though. Not that I see any clear once, but there got to be some that are worth at least checking and debunking – innovation slowing so that you need more slack to innovate at the same rate, or increasing wealth creating more slack thereby decreasing competition that would’ve otherwise kept prices down, etc.

The article was very elucidating, but I’m not quite able to now look at a system and tell whether it needs more or less slack or how to establish a mechanism that could produce that slack. That would be important since I have a number of EA friends who could use some more slack to figure out psychological issues or skill up on some areas. The EA funds try to help a bit here, but I feel like we need more of that.

Denis Drescher's Shortform

Effective Altruism and Free Riding” by Scott Behmer: Personal takeaways

Coordination is an oft-discussed topic within EA, and people generally try hard to behave cooperatively toward other EA researchers, entrepreneurs, and donors present and future. But “Effective Altruism and Free Riding” makes the case that standard EA advice favors defection over cooperation in prisoner’s dilemmas (and stag hunts) with non-EAs. It poses the question whether this is good or bad, and what can be done about it.

I’ve had a few thoughts while reading the article but found that most of them were already covered in the most upvoted comment thread. I’ll still outline them in the following as a reference for myself, to add some references that weren’t mentioned, and to frame them a bit differently.

The project of maximizing gains from moral trade is one that I find very interesting and promising, and want to investigate further to better understand its relative importance and strategic implications.

Still, Scott’s perspective was a somewhat new one for me. He points out that in particular the neglectedness criterion encourages freeriding: Climate change is a terrible risk but we tend to be convinced by neglectedness considerations that additional work on it is not maximally pressing. In effect, we’re freeriding on the efforts of activists working on climate change mitigation.

What was new to me about that is that I’ve conceived of neglectedness as a cheap coordination heuristic. Cheap in that it doesn’t require a lot of communication with other cooperators; coordination in the sense that everyone is working towards a bunch of similar goals but need to distribute work among themselves optimally; and heuristic in that it falls short insofar as values are not perfectly aligned, momentum in capacity building is hard to anticipate, and the tradeoffs with tractability and importance are usually highly imprecise.

So in essence, my simplification was to conceive of the world as filled with agents like me in values that use neglectedness to coordinate their cooperative work, and Scott conceives of the world as filled with agents very much unlike me in values that use neglectedness to freeride off of each other’s work.

Obviously, neither is exactly true, but I don’t see an easy way to home in on which model is better: (1) I suppose most people are not centrally motivated by consequentialism in their work, and it may be impossible for us to benefit the motivations that are central to them. But then again there are probably consequentialist aspects to most people’s motivations. (2) Insofar as there are aspects to people’s motivations for their work that we can benefit, how would these people wish for their preferences to be idealized (if that is even the framing that they’d prefer to think about their behavior)? Caspar Oesterheld discusses the ins and outs of different forms of idealization in the eponymous section 3.3.1 of “Multiverse-wide Cooperation via Correlated Decision Making.” The upshot is, very roughly, that idealization through additional information seems less doubious than idealization through moral arguments (Scott’s article mentions advocacy for example). So would exposing non-EAs to information about the importance of EA causes lead them to agree that people should focus on them even at the expense of the cause that they chose? (3) What consequentialist preferences should be even take into account – only altruistic ones or also personal ones, since personal ones may be particularly strong? A lot of people have personal preferences not to die or suffer and for their children not to die or suffer, which may be (imperfectly) aligned with catastrophe prevention.

But the framing of the article and the comments was also different from the way I conceive of the world in that it framed the issue as a game between altruistic agents with different goals. I’ve so far seen all sorts of nonagents as being part of the game by dint of being moral patients. If instead we have a game between altruists who are stewards of the interests of other nonagent moral patients, it becomes clearer why everyone is part of the game, their power, but there are a few other aspects that elude me. Is there a risk of double-counting the interests of the nonagent moral patients if they have many altruist stewards – and does that make a difference if everyone does it? And should a bargaining solution only take the stewards’ power into account (perhaps the natural default, for better or worse) or also the number of moral patients they stand up for? The first falls short of my moral intuitions in the case. It may also cause Ben Todd and many others to leave the coalition because the gains from trade are not worth the sacrifice for them. Maybe we can do better. But the second option seems gameable (by pretending to see moral patienthood where one in fact does not see it) and may cause powerful cooperators to leave the coalition if they have a particularly narrow concept of moral patienthood. (Whatever the result, it seems like that this the portfolio that commenters mentioned, probably akin to the compromise utility function that you maximize in evidential cooperation – see Caspar Oesterheld’s paper.)

Personally, I can learn a lot more about these questions by just reading up on more game theory research. More specifically, it’s probably smart to investigate what the gains from trade are that we could realize in the best case to see if all of this is even worth the coordination overhead.

But there are probably also a few ways forward for the community. Causal (as opposed to acausal) cooperation requires some trust, so maybe the signal that there is a community of altruists that cooperate particularly well internally can be good if paired with the option of others to join that community by proving themselves to be sufficiently trustworthy. (That community may be wider than EA and called differently.) That would probably take the shape of newcomers making the case for new cause areas not necessarily based on their appeal to utilitarian values but based on their appeal to the values of the newcomer – alongside an argument that those values wouldn’t just turn into some form of utilitarianism upon idealization. That way, more value systems could gradually join this coalition, and we’d promote cooperation the way Scott recommends in the article. It’ll probably make sense to have different nested spheres of trust, though, with EA orgs at the center, the wider community around that, new aligned cooperators further outside, occasional mainstream cooperators further outside yet, etc. That way, the more high-trust spheres remain even if sphere’s further on the outside fail.

Finally, a lot of these things are easier in the acausal case that evidential cooperation in large worlds (ECL) is based on (once again, see Caspar Oesterheld’s paper). Perhaps ECL will turn out to make sufficiently strong recommendations that we’ll want to cooperate causally anyway despite any risk of causal defection against us. This stikes me as somewhat unlikely (e.g., many environmentalists may find ECL weird, so there may never be many evidential cooperators among them), but I still feel sufficiently confused about the implications of ECL that I find it at least worth mentioning.

Load More