Very nice post! As a late-stage CS PhD student, I agree with pretty much everything. I wish more people would read this before deciding whether to get a PhD or not.
One extremely minor thing:
>From what I have heard, these are some problems you might have:
>[...] your supervisor wants to be co-author even if they did nothing to help.
In computer science (at least in AI at top institutions in the US), it is the norm for PhD supervisors to be a co-author on most or all papers that their students write, even if they contribute very little. One can debate wh...
I agree with the argument. If you buy into the idea of evidential cooperation in large worlds (formerly multiverse-wide superrationality), then this argument might go through even if you don't think alien values are very aligned with humans. Roughly, ECL is the idea that you should be nice to other value systems because that will (acausally via evidential/timeless/functional decision theory) make it more likely that agents with different values will also be nice to our values. Applied to the present argument: If we focus more on existential risks that take...
Probably you're already aware of this, but the APA's Goldwater rule seems relevant. It states:
On occasion psychiatrists are asked for an opinion about an individual who is in the light of public attention or who has disclosed information about himself/herself through public media. In such circumstances, a psychiatrist may share with the public his or her expertise about psychiatric issues in general. However, it is unethical for a psychiatrist to offer a professional opinion unless he or she has conducted an examination and has been granted pr...
I would guess there are many other related movements. For instance, I recently found this article about Comte. Much of it also sounds somewhat EA-ish:
...[T]he socialist philosopher Henri de Saint-Simon attempted to analyze the causes of social change, and how social order can be achieved. He suggested that there is a pattern to social progress, and that society goes through a number of different stages. But it was his protégé Auguste Comte who developed this idea into a comprehensive approach to the study of society on scientific principles, which he initia
I agree that altruistic sentiments are a confounder in the prisoner's dilemma. Yudkowsky (who would cooperate against a copy) makes a similar point in The True Prisoner's Dilemma, and there are lots of psychology studies showing that humans cooperate with each other in the PD in cases where I think they (that is, each individually) shouldn't. (Cf. section 6.4 of the MSR paper.)
But I don't think that altruistic sentiments are the primary reason for why some philosophers and other sophisticated people tend to favor cooperation in the prisoner's dilemma again...
A few of the points made in this piece are similar to the points I make here: https://casparoesterheld.com/2017/06/25/complications-in-evaluating-neglectedness/
For example, the linked piece also argues that returns may diminish in a variety of different ways. In particular, it also argues that the returns diminish more slowly if the problem is big and that clustered value problems only produce benefits once the whole problem is solved.
Nice post! I generally agree and I believe this is important.
I have one question about this. I'll distinguish between two different empirical claims. My sense is that you argue for one of them and I'd be curious whether you'd also agree with the other. Intuitively, it seems like there are lots of different but related alignment problems: "how can we make AI that does what Alice wants it to do?", "how can we make AI that does what the US wants it to do?", "how can we make AI follow some set of moral norms?", "how can we make AI build stuff in factories for ... (read more)