AI alignment and game theory researcher.
Nitpicky feedback on the presentation:
If I am understanding it correctly, the current format of the tables makes them fundamentally incapable of expressing evidence for insects being unable to feel pain. (The colour coding goes from green=evidence for to red=no evidence, and how would you express ??=evidence against?) I would be more comfortable with a format without this issue, in particularly since it seems justified to expect the authors to be biased towards wanting to find evidence for. [Just to be clear, I am not pushing against the results, or against for caring about insects. Just against the particular presentation :-).]
After thinking about it more, I would interpret (parts of) the post as follows:
Is this interpretation correct? If so, then I register the complaint that the post is a bit confusing --- not particularly sure why, just noticing that it made me confused. Perhaps it's the thing where I first understood the tables/conclusions as "how much pain do these types of insects feel?". (And I expect others might get similarly confused.)
I saw the line "found no good evidence that anything failed any criterion", but just to check explicitly: What do the confidence levels mean? In particular, should I read "low confidence" as "weak evidence that X feels pain-as-operationalized-by-Criterion Y"? Or as "strong evidence that X does not feel pain-as-operationalized-by-Criterion Y"?
In other words:
"National greatness, and staying ahead of China for world influence requires that we have the biggest economy. To do that, we need more people." -Matt Yglesias, One Billion Americans.Yeah, the guy who has chosen to have one child is going to inspire me to make the sacrifices involved in having four. It might be good for America, but the ‘ask’ here looks like it is that I sacrifice my utility for Matt’s one kid, and thus is not cooperate-cooperate. I’ll jump when you jump.
Two pushbacks here:
(1) The counterargument seems rather weak here, right? Even if Matt Yglesias had no kids, that doesn't mean his argument isn't valid. EG, if a single non-vegan person claims that more people should be vegan, will you view that as evidence that people should not be vegan? ;-) (Not that I disagree with your claim. Just with your argument.)
(2) Did you actually read One Billion Americans, or are you just taking the citation and interpretting it as Matt Yglesias making an argument for people having more children? I didn't read the book, so I am not sure. But I listened to a podcast with Matt Yglesias, about the book, and my impression was that he was primarily arguing for changing the immigration policy, and (if memory serves) not really making any strong claims about how many kids people should have.
Uhm, yes, that seems right. That's why this matters.
Yes, sure, probabilities are only in the map. But I don't think that matters for this. Or I just don't see what argument you are making here.
(CLT is in the map, expectations are taken in the map, and decisions are made in the map (then somehow translated into the territory via actions). I don't see how that says anything about what EV reasoning relies on.)
Agree with Acylhalide's point - you only need to be non-Dutchbookable by bets that you could actually be exposed to.
To address a potential misunderstanding:
I agree with both Sharmake's examples. But they don't imply you have to maximise expected utility always. Just when the assumptions apply.
More generally: expected utility maximisation is an instrumental principle. But it is justified by some assumptions, which don't always hold.
Yes, the expected utility is larger.
The claim is that there is nothing incoherent about not maximising expected utility in this case.
To try rephrasing:
Principle 1: if you have to choose a X% chance of getting some outcome A, and a >=X% chance of a strictly better outcome B, you should take B.
Principle 2: if you will be facing a long series of comparably significant choices, you should decide each of them based on expected utility maximisation.
Principle 3: you should do expected utility maximisation for every single choice. Even if that is the last/most important choice you will ever make.
The claim is that: P1 is solid. P2 follows from P1 (via Central Limit Theorem, or whatever math). But P3 does not follow from P1/P2, and there will be cases where it might be justified to not obey P3. (Like the case with 51% chance of doubling the world's goodness, 49% chance of destroying it.)
Note that I am not claiming it's wrong to do expected utility maximisation in all scenarios. Just saying that it both doing and not doing it is OK. And therefore it is (very?) non-strategic to associate your philosophical movement with it. (Given that most people's intuitions seem to be against it.)
Does this explanation make sense? Maybe I should change the title to something with expected utility?
This gave me an idea for an experiment/argument. Posting here, in case somebody wants come up with a more thought-out version of it and do it.
[On describing what would change his mind:] You couldn’t find weird behaviors [in the AI], no matter how hard you tried.
People like to take an AI, poke it, and then argue "it is doing [all these silly mistakes], therefore [not AGI/not something to worry about/...]". Now, the conclusion might be right, but the argument is wrong --- even dangerous things can be stupid is some settings. Nevertheless, the argument seems convincing.
My prediction is that people make a lot of mistakes that would seem equally laughable if it was AI that made them. Except that we are so used to them that we don't appreciate it. So if one buys the argument above, they should conclude that humans are also [not general intelligence/something to worry about/...]. So perhaps if we presented the human mistakes right, it could become a memorable counterargument to "AI makes silly mistakes, hence no need to worry about it".
Some example formats:
(uhm, or "Quizz: AI or Trump?"; wouldn't mention this, except bots on that guy already exist).
Obligatory disclaimer: It might turn out that humans really don't make [any sorts of] [silly mistakes current AI makes], or make [so few that it doesn't matter]. If you could operationalize this, that would also be valuable.
What is "these mistakes"? I don't know . Exercise for the reader.
Some thoughts: 1) Most importantly: In your planning, I would explicitly include the variable of how happy you are. In particular, if the AI Safety option would result in a break-up of a long-term & happy relationship, or cause you to be otherwise miserable, it is totally legitimate to not do the AI Safety option. Even if it was higher "direct" impact. (If you need an impact-motivated excuse - which might even be true - then think about the indirect impact of avoiding signalling "we only want people who are so hardcore that they will be miserable just to do this job".)
2) My guess: Given that you think your QC work is unlikely to be relevant to AI Safety, I personally believe that (ignoring the effect on you), the AI Safety job is higher impact.
3) Why is it hard to hire world experts to work on this? (Some thoughts, possibly overlapping with what other people wrote.)
4) I am tempted to say that all the people who could be effectively bought with money are already being bought with money, so you donating doesn't help here. But I think a more careful phrasing is "recruiting existing experts is bottlenecked on other things than money (including people coming up with good recruiting strategies)".
5) Phrased differently: In our quest for developing the AI Safety field, there is basically no tradeoff between "hiring 'more junior' people (like you)" and "recruiting senior people", even if those more junior people would go earning to give otherwise.
Two considerations seem very relevant here:(1) Is your primary goal to help Ukranians, or to make this more costly for Russia?(2) Do you think the extra money is likely to change the outcome of the war, or merely the duration?