MichaelA

Senior Research Manager; also guest fund manager @ Rethink Priorities; also EA Infrastructure Fund
Working (0-5 years experience)
10783Oxford, UKJoined Dec 2018

Bio

I’m Michael Aird, a Senior Research Manager at Rethink Priorities and guest fund manager at the Effective Altruism Infrastructure Fund. Opinions expressed are my own. See here for Rethink Priorities' job openings or expression of interest forms, here for a list of EA-aligned funding sources you could apply to,  and here for my top recommended resources for people interested in EA/longtermist research careers. You can give me anonymous feedback here.

With Rethink, I'm mostly focused on co-leading our AI Governance & Strategy team. I also do some nuclear risk research, give input on Rethink's Generalist Longtermism team's work, and do random other stuff.

Previously, I did a range of longtermism-y and research-y things as a Research Scholar at the Future of Humanity Institute, a Summer Research Fellow at the Center on Long-Term Risk, and a Researcher/Writer for Convergence Analysis

I also post to LessWrong sometimes.

If you think you or I could benefit from us talking, feel free to message me! You might also want to check out my post "Interested in EA/longtermist research careers? Here are my top recommended resources".

Sequences
3

Moral uncertainty
Risks from Nuclear Weapons
Improving the EA-aligned research pipeline

Comments
2394

Topic Contributions
763

A list of EA-related podcasts

Radio Bostrom - "Audio narrations of academic papers by Nick Bostrom."

Re-Announcing High Impact Engineers (FKA EA Engineers)

As we've discussed, I'm tentatively excited about HIE's plans, I think changing your name was a good idea, and I think this new name is good.

But I wanted to push back on / hopefully clarify one part of the reasoning (partly because I think I was a/the source for that bit of the reasoning, so maybe I didn't express it clearly when we chatted):

To not monopolise funding for engineering-related EA organisations. This is significant consideration from a funder’s point of view as they may be hesitant to fund another organisation that may help to serve engineers in a different way to ours because they are under the impression that we are already fully serving this space.

I would instead phrase this more like:

To not monopolise the space of engineering-related EA organisations

  • People in/around the EA community often put a lot of emphasis on neglectedness and cooperation. 
  • This basically makes sense, but has the unfortunate consequence that often people are too quick to dismiss an area/idea as "covered" simply because one person/group is working on it or appears to be working on it. 
    • This can lead to missed opportunities, given that the existing project may later fold, may not be highly effective, may only cover part of the space, may have limited capacity, or may take one specific approach to it even though experimentation and multiple approaches would be valuable. 
  • This problem is probably exacerbated when a project/org is called something like "EA [x]".
    • Because that may increase the chance that other people understandably but incorrect assume that that project/org is sufficiently covering the intersection of EA and [x]. 
    • And because then people may worry that that org/project is trying to stake a claim to the whole space, or would feel insulted / confused if another org stepped in to the space.
    • This seems like an argument in favor of names like 80,000 Hours and GiveWell rather than "EA Career Advice" or "Global Health & Development Charity Evaluators".
  • (Another, unrelated way to mitigate this problem is to explicitly and repeatedly (a) mention that you think other projects in the same space could still be valuable, and (b) explain what your org/project expects to do and what people might guess you'd do that you don't expect to do.)

...ok, that phrasing definitely seems too long for the post, but I guess I wanted to unpack the reasoning more fully while I was at it.

So a key thing I'm highlighting here is that I'm not really worried about funders overly assuming orgs called "EA [x]" will sufficiently cover everything at the intersection of EA and [x] and hence not fund other projects; rather, I'm worried that various potential entrepreneurs-or-similar will think that or will worry about what the existing org will think about them starting a new thing. 

I think where funders come in is that they may be less likely to fund an org that is itself going to be called "EA [x]", because that name probably somewhat increases the extent to which this org may crowd out other projects.

(Btw, I can't "take credit" for spotting this consideration; I've seen/heard it discussed by others, and am essentially just a messenger.)

AI timelines via bioanchors: the debate in one place

Thanks, this seems like a useful collection to have made!

Here are some additional places where Ajeya provides a somewhat more conversational/accessible overview or discussion of her report:

EA Opportunities: A new and improved collection of internships, contests, events, and more.

Thanks for your work on this! I imagine this will indeed be valuable. 

We would especially appreciate it if you submitted niche opportunities that would be easy to miss (e.g., Buck’s call for language model investigators, Rethink Priorities Nuclear research projects)

All collections of EA-relevant research questions which I'm aware of and which are publicly shareable are in A central directory for open research questions. So you could harvest that directory for other collections that seem sufficiently well-scoped/high-priority/packaged with a chance of light mentorship that it's worth including them in your site? And/or you could include a link to that directory on the site so people who are interested can poke around themselves based on their own interests, skills, and plans?

(Also, btw, the nuclear risk research projects post is based on work supported by RP but was posted in a personal capacity, and the offer of assistance for people who work on those things is from me rather than from RP. But that distinction is a bit murky and not very important, so it's fine by me if you don't make any edits based on that.)

EA Opportunities: A new and improved collection of internships, contests, events, and more.

I think actually you mean "permanent" vs "temporary", rather than "full-time" vs "part-time". 

I think a lot of the opportunities you do/will/should cover will either will be or can be full-time but just temporary, e.g. GovAI's fellowships are on your board and are intended as full-time, and CERI and SERI's fellowships can be done full-time and probably should be on your board when they next have openings. And I think your target audience is probably often constrained to not taking permanent things yet or not being (demonstrably) skilled enough to get into the relevant permanent things yet, but could do temporary things at a full-time load. 

So I suggested editing the phrasings in light of that in this post and (if the site says relevant things) on the site.

Why EA needs Operations Research: the science of decision making

I think what RP means by that term is "a researcher focused on figuring out & improving various operations-related things", which is closer to industrial/organizational psychology + business administration + some other stuff than to the field of "operations research" ("a discipline that deals with the development and application of advanced analytical methods to improve decision-making [...] considered to be a subfield of mathematical sciences" (Wikipedia)). 

So I think this is just an unfortunate overlap of terminology, rather than us actually doing something fairly similar to what this post is about.

(I guess I blame whoever named the field of operations research, since the name really doesn't feel intuitive, but that ship has probably long since sailed.)

(Unnecessary disclaimer: I work at RP and Peter is my manager, but that's not relevant to me reading this post or leaving this comment)

Enlightenment Values in a Vulnerable World

I think this post contains some major and some minor errors and is overall fairly "one-sided", and that the post will therefore tend to overall worsen & confuse (rather than improve & clarify) debates and readers' beliefs. Below I discuss what I see as some of the errors or markers of one-sidedness in this post. I then close with some other points, e.g. emphasising that I do think good critiques and red-teaming is valuable, noting positives of this post, and acknowledging that this comment probably feels kind-of rude :) 


Here are some of the things I see as issues in this post. Some are in themselves important, and others are in themselves minor but seem to me like indications of the post generally seeming quite inclined to support a given conclusion rather than more neutrally surveying a topic and seeing what it lands on.* I've bolded key points to help people skim this.

  • As Zach mentioned, I think you at least somewhat overstate the extent to which Bostrom is recommending as opposed to analyzing these interventions.
    • Though I do think Bostrom probably could and should have been clearer about this, given that many people have gotten this impression from the paper.
  • You seem to argue (or at least give the vibe that) that there's there's so little value in trying to steer technological development for the better than we should mostly not bother and instead just charge ahead as fast as possible. It seems to me that this conclusion is probably incorrect (though I do feel unsure), that the arguments you've presented for it are somewhat weak, and that you haven't adequately discussed arguments against it.
    • Your arguments for this conclusion include that it's hard to predict the potential benefits and harms of various technologies, that some dangerous and powerful techs like AI can also protect us from other things, and that actors who would steer technological development have motives other than just making the world better.
      • I think these are in fact all true and important points, and it's good for people to consider them.
      • But I think there are still many cases where we can be pretty confident that our best bet is that some tech will reduce risk or will increase it and that some way of steering tech will have net positive effects.
        • I don't mean that we can be confident that this will indeed happen this way, but that we can be confident that even after another 10,000 hours of thinking and research we'd still conclude these actions are net positive in expectation (or net negative, in the cases where that's our guess). And we should take action on that basis.
        • (I won't try to justify this here due to time constraints, and it would be fair to not be convinced. But hopefully readers can try to think of examples and realise for themselves that my stance seems right.)
        • And if we can't be confident of that right now, then it seems to me that we should try to (a) gain greater clarity on what tech steering would be good and greater ability to learn that or implement our learnings effectively, and (b) avoid actively accelerating tech dev in the meantime. (As opposed to treating our inability to usefully steer things as so unchangeable that we should just charge ahead and hope for the best.)
      • It seems odd to me to act as though we should be so close to agnostic about the net benefits or harms of all techs and so close to untrusting of any actors who could steer tech development that we should instead just race ahead as fast as we can in all directions.
      • In some cases, I think making simple models, Fermi estimates, or forecasts could help make "each side"'s claims more clear and help us figure out which should get more weight. An example of what this could look like is here: https://blog.givewell.org/2015/09/30/differential-technological-development-some-early-thinking/ (This actually overall highlights the plausibility of the "maybe accelerating AI is good" stance. And I agree that that's plausible. I'm not saying this post supports my conclusion, just that it seems like an example of a productive way to advance this discussion.)
    • I haven't re-read your post closely to check what your precise claims and are if you somewhere provide appropriate caveats. But I at least think that the impression people would walk away with is something like "we should just race ahead as fast as possible".
  • A core premise/argument in your post appears to be that pulling a black ball and an antidote (i.e., discovering a very dangerous technology and a technology that can protect us from it) at the same time means we're safe. This seems false, and I think that substantially undermines the case for trying to rush forward and grab balls from the urn as fast as possible.
    • I think the key reason this is false is that "discovering" a technology or "pulling a ball from the urn" does not mean it has reached maturity and been deployed globally. So even if we've discovered both the dangerous and protective technology, it's still possible for the dangerous technology to be deployed in a sufficiently bad way before the protective technology has been deployed in a sufficiently good way.
      • I think there are also reasons why that might be likely, e.g. in some ways it seems easier to destroy than to create, and some dangerous technologies would just need to be deployed once somewhere whereas some protective technologies would need to be deployed continuously and everywhere. (That might be the same point stated in two separate ways - not sure.)
      • OTOH, there are also reasons why that might be unlikely, e.g. far more people want to avoid existential catastrophe than to enact it.
      • Overall I'm not sure which is more likely, but it definitely seems at least plausible that we could end up with disaster if we discover both a very dangerous tech and a paired protective tech at the same time.
    • I'll illustrate with one of your own examples: "[Increasing our technological ability] slowly, one ball at a time, just means less chance at pulling antidote technologies in time to disable black ball risks.For example, terraforming technology which allows small groups of humans to make changes to a planet’s atmosphere and geography may increase existential risk until space-settling technology puts people on many planets. If terraforming technology typically precedes space-settling then accelerating the pace of progress reduces risk." But I think if we develop such terraforming technology and such space-settling technology at the same time, or even develop space-settling technology somewhat earlier, that does guarantee we will in fact have built self-sustaining settlements on many places before an individual uses the terraforming technology in a bad way.
      • It's still totally possible for us to all die due to the terraforming technology before those self-sustaining settlements are set up.
    • Another way to illustrate this: You write "If we discovered all possible technologies at once (which in Bostrom’s wide definition of technology in the VWH paper includes ideas about coordination and insight), we would be in the safe region." I encourage readers to genuinely try to imagine that literally tomorrow literally the ~8 billion people who exist collectively discover literally all possible technologies at once, and then consider whether they're confident humanity will exist and be on track to thrive in 2023. Do you (the reader) feel confident that everything will go well in that world where all possible techs and insights on dumped on us at once?
  • I also don't agree, and don't think Bostrom would claim, that technological maturity means having discovered all possible technologies, or that we would necessarily be safe if we'd discovered & deployed all possible technologies (even if we survive the initial transition to that world). 
    • Bostrom writes "By ‘technological maturity’ we mean the attainment of capabilities affording a level of economic productivity and control over nature
      close to the maximum that could feasibly be achieved (in the fullness of time) (Bostrom, 2013)." That phrasing a bit vague, but I think that attaining that level of capabilities doesn't mean that we've actually got all possible technologies or that every given individual has the maximum possible capabilities.
    • It seems plausible/likely that some technologies are sufficiently dangerous that we'll only be safe if we're in a world where them ever being discovered or ever being deployed is prevented - i.e., that no protective measure would be adequate except prevention.
      • iirc, Bostrom's discussion of "Type-0 vulnerabilities" is relevant here.
  • I think the following bolded claim is false, and I think it's very weird to make this empirical claim without providing any actual evidence for it: "AI safety researchers argue over the feasibility of ‘boxing’ AIs in virtual environments, or restricting them to act as oracles only, but they all agree that training an AI with access to 80+% of all human sense-data and connecting it with the infrastructure to call out armed soldiers to kill or imprison anyone perceived as dangerous would be a disaster."
    • I am 100% confident that not all AI safety researchers have even considered that question, let alone formed the stance you suggest they all agree on.
    • Perhaps you meant they "would all agree"? Still though, it would seem odd to be confident of that without providing any justification.
    • And I think in fact many would disagree if asked. In fact, I expect that many of them would believe that what the future should look like would technically or basically involve this happening; we have a properly aligned superintelligent AI that either already has access to those things or could gain access to those things if it simply chose to do so.
  • I think "If it is to fulfill its mission of preventing anthropogenic risk long into the future, the global surveillance state cannot afford to risk usurpation" and related claims are basically false or misleading.
    • It appears to me that we're fairly likely to be in or soon be in a "time of perils", where existential risk is unusually high. There are various reasons to expect this to subside in future besides a global surveillance state. So it seems pretty plausible that it would be best to temporarily have unusually strong/pervasive surveillance, enforcement, etc. for particular types of activities.
    • And if we've set this actor up properly, then it should be focused on what's net positive overall and should not conflate "ensuring this actor has an extremely high chance of maintaining power helps reduce some risks" with "ensuring this actor has an extremely high chance of maintaining power is overall net beneficial".
    • To be clear, I'm not saying that we should do things like this or that it'd work if we tried; I'm just saying that thinking that increased surveillance, enforcement, moves towards global governance, etc. would be good doesn't require thinking that permanent extreme levels (centralised in a single state-like entity) would be good.
  • The following seems like a misrepresentation of Bostrom, and one which is in line with what I perceive as a general one-sidedness or uncharitability or in this post: "Bostrom continues to assume that the power to take a socially beneficial action is sufficient to guarantee that the state will actually do it. “States have frequently failed to solve easier collective action problems … With effective global governance, however, the solution becomes trivial: simply prohibit all states from wielding the black-ball technology destructively.”"
    • That quote does not state that the power to take a socially beneficial action is sufficient to guarantee that a state will actually take it. A solution can be trivial but not taken.
      • Also, the "effective" in "effective global governance" might be adding something beyond "power" along the lines of "this governance is pointed in the right direction"?
    • I haven't read the VWH paper in a while, so maybe he does make this claim elsewhere, or maybe he repeatedly implies it without stating it. But that quote does not seem to demonstrate this.

Some other things I want to make sure I say (not issues with the post):

  • To be clear, I do think it's valuable to critically discuss & red-team the VWH paper in particular and also other ideas and writings that are prominent within longtermism. And I personally wish Bostrom had written the VWH paper somewhat differently, and I don't feel confident that the interventions it discusses are net positive. So this comment is not meant to discourage other critical discussions or to strongly defend the interventions discussed in VWH.
  • But I do think it's important to counter mistaken and misleading posts in general, even if the posts are good-faith and are attempting to play a valuable role of criticizing prominent ideas.
  • I wrote this comment pretty quickly, so I don't fully justify things and my tone is sometimes a bit sharp or uncharitable - apologies in advance for that.
    • (I expect that if the original poster and I instead had a call we would get on the same page faster and feel more positively toward each other, and that I would come across as a bit less rude than this comment might.)
  • I do think there are some good elements of this post (e.g., the writing is generally clear, you include a decent summary at the start, you keep things organized nicely with headings, and some of your points seem true and important). I focus on the negatives since they seem more important and due to time constraints.
  • As a heads up, I'm unlikely to reply to replies to this, since I'm trying to focus on my main work atm.

*To be clear, I'm a fan of red-teaming, which is not neutral surveying but rather deliberately critical. But that should then be framed explicitly as red-teaming. 

Will Aldred's Shortform

I'm also pretty sure this wasn't a totally original brainstorm, and that I was remembering these examples having read something on a similar topic somewhere, probably here on the Forum, though I can't recall which post it was.

Maybe it was some combination of the posts with the megaprojects tag?

Will Aldred's Shortform

Hmm, it feels unclear to me what you're claiming here. In particular, I'm not sure which of the following is your claim:

  1. "Right now all money committed to EA could be spent on things that we currently (should) think are at least slightly net positive in expectation. (Even if we maybe shouldn't spend on those things yet, since maybe we should wait for even better opportunities.)"
  2. "Right now all money committed to EA could be spent on things that might be net positive in expectation. (But there aren't enough identified opportunities that we currently think are net positive to absorb all current EA money. Some of the things currently look net negative but with high uncertainty, and we need to do further research or wait till things naturally become closer and clearer to find out which are net positive. We also need to find more opportunities.)"

1 is a stronger and more interesting claim than 2. But you don't seem to make it clear which one you're saying. 

If 2 is true, then we still are "severely bottlenecked by good funding opportunities" + by strategic clarity. So it might be that the people you're talking to are already thinking 2, rather than that EA funding is effectively infinite? 

To be clear, I do think 2 is importantly different from "we have effectively infinite money", in particular in that it pushes in favor of not spending on extremely slightly net positive funding opportunities now since we want to save money for when we've learned more about which of the known maybe-good huge funding opportunties are good.* So if there are people acting and thinking as though we have effectively infinite money, I do think they should get ~this message. But I think your shortform could maybe benefit from distinguishing 1 and 2.

(Also, a nit-picky point: I'd suggest avoiding phrasing like "could plausibly absorb all EA funding" without a word like "productively", since of course there are things that can literally just absorb our funding - literally just spending is easy.)

*E.g., personally I think trying to spend >$1b in 2023 on each of the AI things you mentioned would probably require spending on some net negative in expectation things, but I also think that we should keep those ideas in mind for future and spend a bit slower on other things for that reason.

Using the “executive summary” style: writing that respects your reader’s time

Cool, glad to hear that.

fwiw, I think a good summary can & should also summarise the reasoning, not only the conclusions. (Though of course it'll still miss many details of both the reasoning and the conclusions.)

I'd also flag that I think this should be some sort of nudge, suggestion, optional box to fill in, or whatever, rather than a required box. So people could still decide to not have a summary if they wanted, e.g. if that would mess with their engaging discursive style in a given piece.

Load More