All of Benjamin Hilton's Comments + Replies

Totally agree! Indeed, there's a classic 80k article about this.

When working out your next steps, we tend to recommend working forwards from what you know, and working backwards from where you might want to end up (see our article on finding your next career steps). We also think people should explore more with their careers (see our article on career exploration).

If there are areas where we're giving the opposite message, I'd love to know – shoot me an email or DM?

Hi Remmelt,

Thanks for sharing your concerns, both with us privately and here on the forum. These are tricky issues and we expect people to disagree about how to about how to weigh all the considerations — so it’s really good to have open conversations about them.

Ultimately, we disagree with you that it's net harmful to do technical safety research at AGI labs. In fact, we think it can be the best career step for some of our readers to work in labs, even in non-safety roles. That’s the core reason why we list these roles on our job board.

We argue for this p... (read more)

Most of our advice on actually having an impact — rather than building career capital — is highly relevant to mid-career professionals. That's because they're entering their third career stage (https://80000hours.org/career-guide/career-planning/#three-career-stages), i.e. actually trying to have an impact. When you’re mid-career, it's much more important to appropriately:

• Pick a problem
• Find a cost-effective way of solving that problem that fits your skills
• Avoid doing harm

So we hope mid-career people can get a lot out of reading our articles. I'd probably i... (read more)

By "engagement time" I mean exactly "time spent on the website".

Thanks for this comment Tyler!

To clarify what I mean by unknown unknowns, here's a climate-related example: We're uncertain about the strength of various feedback loops, like how much warming could be produced by cloud feedbacks. We'd then classify "cloud feedbacks" as a known unknown. But we're also uncertain about whether there are feedback loops we haven't identified. Since we don't know what these might be, these loops are unknown unknowns. As you say, the known feedback loops don't seem likely to warm earth enough to cause a complete destruction of ci... (read more)

I don't currently have a confident view on this beyond "We’re really not sure. It seems like OpenAI, Google DeepMind, and Anthropic are currently taking existential risk more seriously than other labs."

But I agree that if we could reach a confident position here (or even just a confident list of considerations), that would be useful for people — so thanks, this is a helpful suggestion!

Thanks, this is an interesting heuristic, but I think I don't find it as valuable as you do. 

First, while I do think it'd probably be harmful in expectation to work at leading oil companies / at the Manhattan project, I'm not confident in that view — I just haven't thought about this very much.

Second, I think that AI labs are in a pretty different reference class from oil companies and the development of nuclear weapons.

Why? Roughly:

1. Whether, in a broad sense, capabilities advances are good or bad is pretty unclear. (Note some capabilities advances in

Hi Yonatan,

I think that for many people (but not everyone) and for many roles they might work in (but not all roles), this is a reasonable plan.

Most importantly, I think it's true that working at a top AI lab as an engineer is one of the best ways to build technical skills (see the section above on "it's often excellent career capital"). 

I'm more sceptical about the ability to push towards safe decisions (see the section above on "you may be able to help labs reduce risks").

The right answer here depends a lot on the specific role. I think it's importa... (read more)

The Portuguese version at 80000horas.com.br is a project of Altruísmo Eficaz Brasil. We often give people permission to translate our content when they ask - but as to when, that would be up to Altruísmo Eficaz Brasil! Sorry I can't give you a more concrete answer.

(Personal views, not representing 80k)

My basic answer is "yes".

Longer version:

I think this depends what you mean.

By "longtermism", I mean the idea that improving the long-run future is a key moral priority. By "longtermist" I mean someone who personally identifies with belief in longtermism.

I think x-risks are the most pressing problems from a cause-neutral perspective (although I'm not confident about this, there are a number of plausible alternatives, including factory farming).

I think longtermism is also (approximately) true from a cause neutral perspec... (read more)

There are important reasons to think that the change by the EA community is within the measurement error of these surveys, which makes this less noteworthy.

(Like say you put +/- 10 years and +/- 10% on all these answers - note there are loads of reasons why you wouldn't actually assess the uncertainty like this, (e.g. probabilities can't go below 0 or above 1), but just to get a feel for the uncertainty this helps. Well, then you get something like:

•  10%-30% chance of TAI by 2026-2046
• 40%-60% by 2050-2070
• and 75%-95% by 2100

Then many many EA timelines an... (read more)

Thanks for this! Looks like we actually roughly agree overall :)

Thanks for this thoughtful post! I think I stand by my 1 in 10,000 estimate despite this.

A few short reasons: 

• Broad things: First, these scenarios and scenarios like them are highly conjunctive (many rare things need to happen), which makes any one scenario unlikely (although of course there may be many such scenarios). Second, I think these and similar scenarios are reason to think there may be a large catastrophe, but large and existential are a long way apart. (I discuss this a bit here but don't come to a strong overall conclusion. More work on th

Hi! Wanted to follow up as the author of the 80k software engineering career review, as I don't think this gives an accurate impression. A few things to say:

• I try to have unusually high standards for explaining why I believe the things I write, so I really appreciate people pushing on issues like this.
   
• At the time, when you responded to <the Anthropic person>, you said "I think <the Anthropic person> is probably right" (although you added "I don't think it's a good idea to take this sort of claim on trust for important career prioriti

This looks really cool, thanks Tom!

I haven't read the report in full (just the short summary) - but I have some initial scepticism, and I'd love to answers to some of the following questions, so I can figure out how much evidence this report is on takeoff speeds. I've put the questions roughly in order of subjective importance to my ability to update:

• Did you consider Baumol effects, the possibility of technological deflation, and the possibility of technological unemployment, how they affect the profit incentive as tasks are increasingly automated? [My gue

I agree with (a). I disagree that (b) is true! And as a result I disagree that existing CEAs give you an accurate signpost.

Why is (b) untrue? Well, we do have some information about the future, so it seems extremely unlikely that you won't be able to have any indication as to the sign of your actions, if you do (a) reasonably well.

Again, I don't purely mean this from an extreme longtermist perspective (although I would certainly be interested in longtermist analyses given my personal ethics). For example, simply thinking about population changes in the abo... (read more)

Sure, happy to chat about this!

Roughly I think that you are currently not really calculating cost-effectiveness. That is, whether you're giving out malaria nets or preventing nuclear war, almost all of the effects of your actions will be affecting people in the future.

To clarify, by "future" I don't necessarily mean "long run future". Where you put that bar is a fascinating question. But focusing on current lives lost seems to approximately ignore most of the (positive or negative) value, so I expect your estimates to not be capturing much about what matte... (read more)

I'm curious about the ethical decisions you've made in this report. What's your justification for evaluating current lives lost? I'd be far more interested in cause-X research that considers a variety of worldviews, e.g. a number of different ways of evaluating the medium or long-term consequences of interventions.

I agree that I'd love to see more work on this! (And I agree that the last story I talk about, of a very fast takeoff AI system with particularly advanced capabilities, seems unlikely to me - although others disagree, and think this "worst case" is also the most likely outcome.)

It's worth noting again though that any particular story is unlikely to be correct. We're trying to forecast the future, and good ways of forecasting should feel uncertain at the end, because we don't know what the future will hold. Also, good work on this will (in my opinion) give ... (read more)

That particular story, in which I write "one day, every single person in the world suddenly dies", is about a fast takeoff self-improvement scenario. In such scenarios, a sudden takeover is exactly what we should expect to occur, and the intermediate steps set out by Holden and others don't apply to such scenarios. Any guessing about what sort of advanced technology would do this necessarily makes the scenario less likely, and I think such guesses (e.g. "hypnodrones") are extremely likely to be false and aren't useful or informative.

For what it's worth, I ... (read more)

Yeah, it’s a good question! Some thoughts:

• I’m being quite strict with my definitions. I’m only counting people working directly on AI safety. So, for example, I wouldn’t count the time I spent writing this profile on AI (or anyone else who works at 80k for that matter). (Note: I do think lots of relevant work is done by people who don’t directly work on it) I’m also not counting people who think of themselves as on an AI safety career path and are, at the moment, skilling up rather than working directly on the problem. There are some ambiguities, e.g. a

Hi Gideon,

I wrote the 80,000 Hours problem profile on climate change. Thank you so much for this feedback! I’m genuinely really grateful to see such engagement with the things I write - and criticism is always a welcome contribution to making sure that I’m saying the right things.

Just to be clear, when I said “we think it’s potentially harmful to do work that could advance solar geoengineering”, I meant that (with a fair degree of uncertainty), it could be harmful to do work that advances the technology (which I think you agree with) not that all research ... (read more)

I think these are all great points! We should definitely worry about negative effects of work intended to do good. 

That said here are two other places where maybe we have differing intuitions:

• You seem much more confident than I am that work on AI that is unrelated to AI safety is in fact negative in sign. 
• It seems hard to conclude that the counterfactual where any one or more of "no work on AI safety / no interpretability work / no robustness work / no forecasting work" were true is in fact a world with less x-risk from AI overall. That is, while

Ah thanks :) Fixed. 

Yes, there was!

This is a great story! Good motivational content.

But I do think, in general, a mindset of "only I can do this" is innacurate and has costs. There are plenty of other people in the world, and other communities in the world, attempting to do good, and often succeeding. I think EAs have been a small fraction of the success in reducing global poverty over the last few decades, for example.

Here are a few plausible costs to me:

• Knowing when and why others will do things significantly changes estimates of the marginal value of acting. For example, if you are st

This does seem to be an important dynamic.

Here are a few reasons this might be wrong (both sound vaguely plausible to me):

1. If someone being convinced of a different non-weird version of an argument makes it easier to convince them of the actual argument, you end up with more people working on the important stuff overall.
2. If you can make things sound less weird without actually changing the content of what you're saying, you don't get this downside (This might be pretty hard to do though.)

(1) is particularly important if you think this "non-weird to weird" ap... (read more)

That's not the intention, thanks for pointing this out!

To clarify, by "route", I mean gaining experience in this space through working on engineering roles directly related to AI. Where those roles are not specifically working on safety, it's important to try to consider any downside risk that could result from advancing general AI capabilities (this in general will vary a lot across roles and can be very difficult to estimate).

A bit of both - but you're right, I primarily meant "secure" (as I expect this is where engineers have something specific to contribute).