All of gavento's Comments + Replies

Or did you mean ...?
 A dog sitting with his coffee. Flames in the background. Foreground is full of flowers and flowery bushes. A comics draw...

yes. The fire is in an entirely different room.

Thanks for the write-up! I appreciate the orientation towards simplicity and directness of value quantification (including uncertainty) and the overall mindset and spirit of the processes that this induces in people's minds.

I am wondering why I am not actually surprised that I don't see people (esp. EAs, and including  myself) doing this more.
I would expect some of the (assumed) reasons to be simple to state and (in principle) overcome (e.g. lack of skills, not as valued in culture, inadequate epistemic habits, ...) and some to be complex and hard (bi... (read more)

An ingenious friend just pointed out a likely much larger point of influence of quantum particle-level noise on humanity: the randomness in DNA recombination during meiosis (gamete formation) is effectively driven by single molecular machine and the individual crossovers etc likely strongly depends on the Brownian-level noise. This would mean that some substantial part of people would have slightly different genetic makeup, from which I would expect substantial timeline divergence over 100s of years at most (measuring differences on the level of human society).

2
Ozzie Gooen
2y
My impression is that you're arguing that quantum randomness creates very large differences between branches. However, couldn't it still be the case that even more differences would be preferable? I'm not sure how much that first argument would impact the expected value of trying to create even more divergences. 
2
NunoSempere
2y
Yeah, this makes sense, thanks.

For example, if you went back in time 1000 years and painted someone's house a different color, my probability distribution for the weather here and now would look like the historical average for weather here, rather than the weather in the original timeline.

I think the crux here may not be the butterfly effect, but the overall accumulated effect of (quantum) randomness: I would expect that if you went 1000 years back and just "re-ran" the world from the same quantum state (no house painting etc.), the would would be different (at least on the human-precei... (read more)

3
Zach Stein-Perlman
2y
Good point, and that's a crux between two not-unreasonable positions, but my intuition is that even if the universe was deterministic, if you (counterfactually) change house color, the day-to-day weather 1000 years later has essentially no correlation between the two universes.

Re 1.: Yeah, if you consider "determined but unknown" in place of the "non-quantum randomness", this is indeed different. Let me sketch a (toy) example scenario for that: 

We have fixed two million-bit numbers A and B (not assuming quantum random, just fixed arbitrary; e.g. some prefix of pi and e would do). Let P2(x) mean "x is a product of exactly 2 primes". We commit to flipping a quantum coin and on heads, we we destroy humanity iff P2(A), on tails, we destroy humanity iff P2(B). At the time of coin-flip, we don't know P2(A) or P2(B)  (and ass... (read more)

5
gavento
2y
An ingenious friend just pointed out a likely much larger point of influence of quantum particle-level noise on humanity: the randomness in DNA recombination during meiosis (gamete formation) is effectively driven by single molecular machine and the individual crossovers etc likely strongly depends on the Brownian-level noise. This would mean that some substantial part of people would have slightly different genetic makeup, from which I would expect substantial timeline divergence over 100s of years at most (measuring differences on the level of human society).

I think this somehow assumes two types of randomness: quantum randomness and some other (non-quantum) randomness, which I think is an inconsistent assumption.
In particular, in a world with quantum-physics, all events either depend on "quantum" random effects or are (sufficiently) determined*. If e.g. extinction is uncertain, then there will be further splits, some branches avoiding extinction. In the other (although only hypothetical) case of it being determined in some branch, the probability is either 0 or 1, voiding the argument.

*) In a world with quant... (read more)

4
NunoSempere
2y
1. No, I think that the argument goes through if outcomes are deterministic except for quantum-randomness (e.g., worlds were going to either go extinct or not, you just don't know which is the case.) * Or, suppose that in one case I'm talking about quantum randomness and in the other one I'm talking about Bayesian uncertainty. 2. Yes, I think that the "one world survives anyways" argument makes this proposal less exciting. But e.g., worlds in which quantum fluctuations get large enough to avoid an x-risk seem fairly weird, particularly if some x-risks are basins of attraction.

My intuition is also that the discount for academia solving core alignment problems should be (much?) higher than here. At the same time I agree that some mainstream work (esp. foundations) does help current AI alignment research significantly. I would expect (and hope) more of this to still appear, but to be increasingly sparse (relative to amount of work in AI).

I think that it would be useful to have a contribution model that can distinguish (at least) between a) improving the wider area (including e.g. fundamental models, general tools, best practices, ... (read more)