Balliol tends to have a lot of philosophy graduate students, and Wadham is considered to be one of the most left-wing colleges. Looking at the list of current Oxford philosophy graduate students, I noticed there are a lot at St Anne's right now as well. But this can change depending on the year, and philosophy student obviously doesn't mean EA. I would be surprised if any college reliably had a higher number of EAs. AlasdairGives' suggestion to consider funding options makes sense, though you should also keep in mind that the wealthiest colleges get the most applications, so if you apply to St John's, there's more of a risk they won't pick you, and then there's more randomness in the college you end up at.
I had a similar question myself. It seems like believing in a "long reflection" period requires denying that there will be a human-aligned AGI. My understanding would have been that once a human-aligned AGI is developed, there would not be much need for human reflection—and whatever human reflection did take place could be accelerated through interactions with the superintelligence, and would therefore not be "long." I would have thought, then, that most of the reflection on our values would need to have been completed before the creation of an AGI. From what I've read of The Precipice, there is no explanation for how a long reflection is compatible with the creation of a human-aligned AGI.
A lot of good ideas here!
In interested in how Demeny Voting is expected to work psychologically. I would expect just about everyone who is given a second vote (which they are told to submit on behalf of future generations) to use that second vote as a second vote for whatever their first vote was for. I imagine they would either think their first vote was for the best policy/person, in which case they could convince themselves that's best for future generations too, or they would realize their first vote is only good for the short term, but they would double vote for short-termism anyway because that's what matters to them. Either way, I wouldn't expect them to vote against their own preferences with the second vote for future generations. I wouldn't expect many people to vote in some way that they saw as directly good for themselves, and then to cast a second contradictory vote that they saw as possibly neutral or bad for themselves but good for future people.
The system could be set up so they cannot vote for the same thing twice, but then I would expect most people to not use their second vote, unless there were at least two options they happened to like. (To prevent this, it could be set up so everyone voting was required to vote twice, for two different options, but then people may be more likely to just stay home when there's not two options they like. Maybe then there could be fines for not voting, but fining people for not voting for a candidate they don't like might lead to resentment.) However, people being required to vote for more than one option could be interesting as a version of approval voting in which everyone has two totally separate votes for different candidates.
If double voting were allowed, I would expect most people to do exactly that, but framing the second vote as allegedly on behalf of future generations could at least get people thinking more about the longterm, which might change which candidate some people double vote for. Is this sort of indirect effect what you'd be going for with this?
I haven't read the Srinivasan, Gray, and Nussbaum critiques. However, I did read the Krishna critique, and that one uses another rhetorical technique (aside from the sneering dismissal McMahan mentions) to watch out for in critiques of effective altruism. The technique is for the critic of EA to write in as beautiful, literary and nuanced a way possible, in part to subtly frame the EA critic as a much more fully developed, artistic and mature human than the (implied) shallow utilitarian robots who devote their lives to doing a lot of good.
Effective altruism can then be rejected, not on the the basis of logic or anything like that (in fact, caring too much about this kind of logic would be evidence of your lack of humanity), but on grounds that rejecting EA goes along with being nuanced, sophisticated, socially wise, and truly human.
I forgot to mention that your post did help to clarify points and alleviate some of my confusion. Particularly the idea that an ultra-powerful AI tool (which may or may not be sentient) "would still permit one human to wield power over all others."
The hypothetical of an AI wiping out all of humanity because it figures out (or thinks it figures out) that it will increase overall utility by doing so is just one extreme possibility. There must be a lot of credible seeming scenarios opposed to this one in which an AI could be used to increase overall suffering. (Unless the assumption was that a super intelligent being or device couldn't help but come around to a utilitarian perspective, no matter how it was initially programmed!)
Also, like Scott Alexander wrote on his post about this, x-risk reduction is not all about AI.
Still, from a utilitarian perspective, it seems like talking about "AI friendliness" should mean friendliness to overall utility, which won't automatically mean friendliness to humanity or human rights. But again, I imagine plenty of EAs do make that distinction, and I'm just not aware of it because I haven't looked that far into it. And anyway, that's not a critique of AI risk being a concern for EAs; at most, it's a critique of some instances of rhetoric.
I haven't explored the debate over AI risk in the EA movement in depth, so I'm not informed enough to take a strong position. But Kosta's comment gets at one of the things that has puzzled me -- as basically an interested outsider -- about the concern for x-risk in EA. A very strong fear of human extinction seems to treat humanity as innately important. But in a hedonic utilitarian framework, humanity is only contingently important to the extent that the continuation of humanity improves overall utility. If an AI or AIs could improve overall utility by destroying humanity (perhaps after determining that humans feel more suffering than pleasure overall, or that humans cause more suffering than pleasure overall, or that AIs feel more pleasure and less suffering than humans and so should use all space and resources to sustain as many AIs as possible), then hedonic utilitarians (and EAs, to the extent that they are hedonic utilitarians) should presumably want AIs to do this.
I'm sure there are arguments that an AI that destroys humanity would end up lowering utility, but I don't get the impression that x-risk-centered EAs only oppose the destruction of humanity if it turns out humanity adds more pleasure to the aggregate. I would have expected to see EAs arguing something more like, "Let's make sure an AI only destroys us if destroying us turns out to raise aggregate good," but instead the x-risk EAs seem to be saying something more like, "Let's make sure an AI doesn't destroy us."
But maybe the x-risk-centered EAs aren't hedonic utilitarians, or they mostly tend to think an AI destroying humanity would lower overall utility and that's why they oppose it, or there's something else that I'm missing – which is probably the case, since I haven't investigated the debate in detail.