All of Caspar Oesterheld's Comments + Replies

Nice post! I generally agree and I believe this is important.

I have one question about this. I'll distinguish between two different empirical claims. My sense is that you argue for one of them and I'd be curious whether you'd also agree with the other. Intuitively, it seems like there are lots of different but related alignment problems: "how can we make AI that does what Alice wants it to do?", "how can we make AI that does what the US wants it to do?", "how can we make AI follow some set of moral norms?", "how can we make AI build stuff in factories for ... (read more)

Very nice post! As a late-stage CS PhD student, I agree with pretty much everything. I wish more people would read this before deciding whether to get a PhD or not.

One extremely minor thing:
>From what I have heard, these are some problems you might have:
>[...] your supervisor wants to be co-author even if they did nothing to help.
In computer science (at least in AI at top institutions in the US), it is the norm for PhD supervisors to be a co-author on most or all papers that their students write, even if they contribute very little. One can debate wh... (read more)

I agree with the argument. If you buy into the idea of evidential cooperation in large worlds (formerly multiverse-wide superrationality), then this argument might go through even if you don't think alien values are very aligned with humans. Roughly, ECL is the idea that you should be nice to other value systems because that will (acausally via evidential/timeless/functional decision theory) make it more likely that agents with different values will also be nice to our values. Applied to the present argument: If we focus more on existential risks that take... (read more)

I use Pocket Casts and I couldn't find it there. Apparently one can submit to their database here. Also: Is there an RSS feed for the podcast?

3
Pablo
2y
I just added it to Pocket Casts. You should now be able to find it. You can find an RSS feed, as well as links to other podcast platforms, here. (Note that the podcast may not yet be available on some platforms, but will soon.)

Probably you're already aware of this, but the APA's Goldwater rule seems relevant. It states:

On occasion psychiatrists are asked for an opinion about an individual who is in the light of public attention or who has disclosed information about himself/herself through public media. In such circumstances, a psychiatrist may share with the public his or her expertise about psychiatric issues in general. However, it is unethical for a psychiatrist to offer a professional opinion unless he or she has conducted an examination and has been granted pr
... (read more)

I would guess there are many other related movements. For instance, I recently found this article about Comte. Much of it also sounds somewhat EA-ish:

[T]he socialist philosopher Henri de Saint-Simon attempted to analyze the causes of social change, and how social order can be achieved. He suggested that there is a pattern to social progress, and that society goes through a number of different stages. But it was his protégé Auguste Comte who developed this idea into a comprehensive approach to the study of society on scientific principles, which he initia

... (read more)

I agree that altruistic sentiments are a confounder in the prisoner's dilemma. Yudkowsky (who would cooperate against a copy) makes a similar point in The True Prisoner's Dilemma, and there are lots of psychology studies showing that humans cooperate with each other in the PD in cases where I think they (that is, each individually) shouldn't. (Cf. section 6.4 of the MSR paper.)

But I don't think that altruistic sentiments are the primary reason for why some philosophers and other sophisticated people tend to favor cooperation in the prisoner's dilemma again... (read more)

1
JamesDrain
6y
Newcomb's problem isn't a challenge to causal decision theory. I can solve Newcomb's problem by committing to one-boxing in any of a number of ways e.g. signing a contract or building a reputation as a one-boxer. After the boxes have already been placed in front of me, however, I can no longer influence their contents, so it would be good if I two-boxed if the rewards outweighed the penalty e.g. if it turned out the contract I signed was void, or if I don't care about my one-boxing reputation because I don't think I'm going to play this game again in the future. The "wishful thinking" hypothesis might just apply to me then. I think it would be super cool if we could spontaneously cooperate with aliens in other universes. Edit: Wow, ok I remember what I actually meant about wishful thinking. I meant that evidential decision theory literally prescribes wishful thinking. Also, if you made a copy of a purely selfish person and then told them of the fact, then I still think it would be rational to defect. Of course, if they could commit to cooperating before being copied, then that would be the right strategy.

A few of the points made in this piece are similar to the points I make here: https://casparoesterheld.com/2017/06/25/complications-in-evaluating-neglectedness/

For example, the linked piece also argues that returns may diminish in a variety of different ways. In particular, it also argues that the returns diminish more slowly if the problem is big and that clustered value problems only produce benefits once the whole problem is solved.

1
Arepo
6y
Just read this. Nice point about future people. It sounds like we agree on most of this, though perhaps with differing emphasis - yy feeling is that neglectedness such a weak heuristic that we should abandon it completely, and at the very least avoid making it a core part of the idea of effective altruism. Are there cases where you would still advocate using it?
3
Benjamin_Todd
6y
That's a good piece - we thought about many of these issues when working on the framework, and I agree it's not all clearly explained on the page.