Topic Contributions


US Policy Careers Speaker Series - Summer 2022

Hi, thanks for your question! Unfortunately, there are tricky tradeoffs here - to respect the privacy of speakers and to allow them to speak freely, the talks will not be recorded.

Things usually end slowly

As an additional potential analogy, some scenarios people discuss seem analogous to coups. If that's a good analogy, I think it suggests that things would be quick.

AGI Ruin: A List of Lethalities

It looks like the other comments have already offered a good amount of relevant reading material, but in case you're up for some more, I think the ideas expressed in this paper (video introduction here) are a big part of why some people think that we don't know how to train models to have any (somewhat complex) objectives that we want them to have, which is a response to points (1), partly (3), and also (2) (if we interpret the quote in (2) as described in Rob's comment).

This report (especially pp. 1-8) might also make the potential difficulty of penalizing deception more intuitive.

Terminate deliberation based on resilience, not certainty

Ah sorry, I meant to use "volatility" to refer to something like "expected variance in one's estimate of their future beliefs," which is maybe what you refer to as "beliefs about volatility."

Terminate deliberation based on resilience, not certainty

(I'm understanding your comment as providing an example of a situation where volatility goes to 0 with additional evidence.)

I agree it's clear that this happens in some situations -- it's less immediately obvious to me whether this happens in every possible situation.

(Feel free to let me know if I misread. I'm also not sure what you mean by "like this.")

Terminate deliberation based on resilience, not certainty

Thanks for this! I wonder how common or rare the third [edit: oops, meant "fourth"] type of graph is. I have an intuition that there's something weird or off about having beliefs that act that way (or thinking you do), but I'm having trouble formalizing why. Some attempts:

  • If you think you're at (say) the upper half of a persistent range of volatility, that means you expect to update downward as you learn more. So you should just make the update proactively, bringing your confidence toward medium (and narrowing volatility around medium confidence).
    • Special case (?): if you're reading or hearing a debate and your opinion keeps wildly swinging back and forth, at some point you should probably think, "well, I guess I'm bad at evaluating these arguments; probably I should stop strongly updating based on whether I find them compelling."
  • For many estimators, variance decreases as you get more independent samples.
  • At the (unrealistic) limit of deliberation, you've seen and considered everything, and then there's no more room for volatility.
Michael Nielsen's "Notes on effective altruism"

[Cross-posting from the comments]

I want to express two other intuitions that make me very skeptical of some alternatives to effective altruism.

As you write:

"By their fruits ye shall know them" holds for intellectual principles, not just people. If a set of principles throws off a lot of rotten fruit, it's a sign of something wrong with the principles, a reductio ad absurdum.

I really like this thought. We might ask: what are the fruits of ineffective altruism? Admittedly, much good. But also the continued existence, at scale, of extreme poverty, factory farming, low-probability high-impact risks, and threats to future generations, long after many of these problems could have been decimated if there had been widespread will to act.

That's a lot of rotten fruit.

In practice, by letting intuition lead, ineffective altruism systematically overlooks distant problems, leaving their silent victims to suffer and die in masses. So even if strong versions of EA are too much, EA in practice looks like desperately needed corrective movement. Or at least, we need something different from whatever allows such issues to fester, often with little opposition.

Lastly, one thing that sometimes feels a bit lost in discussions of effectiveness: "effective" is just half of it. Much of what resonates is the emphasis on altruism (which is far from unique, but also far from the norm)--on how, in a world with so much suffering, much of our lives should be oriented around helping others.

U.S. EAs Should Consider Applying to Join U.S. Diplomacy

Thanks for writing this! I'd be curious to hear you expand on a few points.

If you’re holding the biosecurity portfolio in China when a crisis strikes, you might suddenly find yourself directly shaping the decisions of the president.

My naive guess would have been that, when a crisis strikes and senior national security officials put their eye on something, junior staff get mostly swept aside (i.e., might get listened to for information about what's happening locally but wouldn't have much room to set priorities or policy goals), because they're (perhaps inappropriately) seen as insufficiently aware of broader political/geopolitical/intelligence considerations, as insufficiently aligned with the President's priorities, and/or as rivals in an opportunity to show impressive leadership ability.

But it sounds like that's not what happens--which of my above assumptions are mistaken, or what are they missing?

Third, a career in the Foreign Service offers you the best possible crash course in international affairs.

Why do you see it as a better crash course than working in U.S.-based international affairs roles (e.g., some U.S.-based roles in the State Department or National Security Council)? (As you mention, there are upsides from being on the front lines, but I could also imagine some U.S.-based roles offering more breadth and more insight into high-level foreign policymaking?)

Also, could you share a bit about how you see the network benefits of working in the Foreign Service comparing to the network benefits of U.S.-based policy roles? Is this a significant upside/downside?

AGI Risk: How to internationally regulate industries in non-democracies

I haven't had time to read this carefully, but regarding funding, this list of EA funding opportunities compiled by Michael Aird might be useful. In case you haven't yet come across these, there are some grants for self-study and independent research in this space, and there are a few grants-based mentorship programs--grants might be easier (in terms of overhead, and also better for the dissemination of your research) than charging readers for downloading a report.

(Oh and since you asked about writing clarity - personally, I usually find a summary at the top really helpful for following a piece of writing.)

Load More