Animal welfare research intern at Charity Entrepreneurship, organizer for Effective Altruism Waterloo, and taking courses in economics.

Earned to give in deep learning at a startup for 2 years for animal charities, and now trying to get into effective animal advocacy research. Curious about s-risks.

Suffering-focused, anti-speciesist, prioritarian, consequentialist. Also, I like math and ethics.

My shortform.


Is this a good way to bet on short timelines?

I guess a few quick responses to each, although I haven't read through your links yet.

  1. I think agenty systems in general can still be very limited in how competent they are, due to the same data/training bottlenecks, even if you integrate a non-agential AGI into the system.
  2. I did see Ajeya's post and read Rohin's summary. I think there might not be any one most reasonable prior for compute necessary for AGI (or whether hitting some level of compute is enough, even given enough data or sufficiently complex training environments), since this will need to make strong and basically unjustified assumptions about whether current approaches (or the next approaches we will come up with) can scale to AGI. Still, this doesn't mean AGI timelines aren't short; it might just means you should do a sensitivity analysis on different priors when you're thinking of supporting or doing certain work. And, of course, they did do such a sensitivity analysis for the timeline question.
  3. In response to this specifically, "As for whether we'd shut it off after we catch it doing dangerous things -- well, it wouldn't do them if it thought we'd notice and shut it off. This effectively limits what it can do to further its goals, but not enough, I think.", what other kinds of ways do you expect it would go very badly? Is it mostly unknown unknowns?
The Comparability of Subjective Scales

It seems to me that a full defense of cardinality and comparability across humans should mention neuroscience, too. For example, we know that brain sizes differ (in total and in regions involved in hedonic experiences) in certain systematic ways, e.g. across ages and between genders. However, these differences are mostly small (brain size is pretty stable after adolescence, although there are still major changes up until 25-30 years old), and we might assume that differences in intensity of experience scale at most roughly 1:1 with size/connectivity and number of neurons firing (in the relevant regions), and while I think this is more likely to be true than not, I'm still not confident in such an assumption.

However, we might also expect some brains to just be more sensitive than others, without expressed behaviour and reports matching this. For example, if two people say they're having 7/10 pains (to an experience with the same physical intensity, e.g. touching something very cold, at the same temperature), but one person's brain regions involved in the (negative) affective experience of pain is far more active, then I would guess that person is having a more negative experience. It would be worth checking research on this. I guess this might be relevant, although it apparently doesn't track the affective component of pain.

Is this a good way to bet on short timelines?

I think I'd need to read more before we could have a very productive conversation. If you want to point me to some writing that you found most persuasive for short timelines (or you could write a post laying out your reasoning, if you haven't already; this could get more useful community discussion, too), that would be helpful. I don't want to commit to anything yet, though. I'm also not that well-read on AI safety in general.

I guess a few sources of skepticism I have now are:

  1. Training an agent to be generally competent in interactions with humans and our systems (even virtually, and not just in conversation) could be too slow or require more complex simulated data than is feasible. Maybe a new version of GPT will be an AGI but not an agent and that might come soon, and while that could still be very impactful, it might not pose an existential risk. Animals as RL agents have had millions of years of evolution to have strong priors fitted to real world environments built into each individual.

  2. I'm just skeptical about trying to extrapolate current trends to AGI.

  3. On AI risk more generally, I'm skeptical that an AI could acquire and keep enough resources without the backing of people with access to them to be very dangerous. It would have to deceive us at least until it's too late for us to cut its access (and I haven't heard of such a scenario that wasn't far-fetched), e.g. by cutting the power or internet, which we can do physically, including by bombing. If we do catch it doing something dangerous, we will cut access. It would need access to powerful weapons to protect its access to resources or do much harm before we could cut its access to resources. This seems kind of obvious, though, so I imagine there are some responses from the AI safety community.

Introducing the EA Funds

I think it's unfortunate we used the word "importance" for one of the factors, since it could also be understood to mean overall how valuable it is to work on something. I think many use the word "scale" now instead for the factor.

If you prioritized by scale only, then you can make a problem arbitrarily large in scale, to the point of uselessness, e.g. "prevent all future suffering".

Presumably wild animal suffering is also much greater in scale than factory farming (or at least the suffering of the farmed animals, setting other effects aside), but it receives much less support since, in part, so far, it seems much less tractable. (Wild animal welfare is still a legitimate cause, though, and it does get support. Wild Animal Initiative was just recommended as a top charity by Animal Charity Evaluators.)

Introducing the EA Funds

If there aren't any good interventions (including researching the problem further to identify good direct interventions), then presumably the cause isn't so important; it would rate low on the tractability scale.

Maybe a weird corner case is saving/investing to donate to the cause later?

I think 1) and 2) are basically backwards. You should support whichever interventions are most effective, regardless of cause, and if these happen to fall into one cause, then that's the most important cause.

Summary of "The Most Good We Can Do or the Best Person We Can Be?" - a Critique of EA

Furthermore, taken to the extreme, they are worried about a scenario where all countries are only paying for offsetting without actual effort for mitigating GHG.

If each country fully offsets its emissions, there'd be no net emissions left, right? Offsets might become more expensive as more do it.

How can I bet on short timelines?

Earmark donations to AI safety orgs/grantmakers for short timeline work. There might be issues with counterfactuals/fungibility you'll need to talk through with them.

How can I bet on short timelines?

Start a fund/grantmaking organization to pool funds with others to support short timeline projects? You might be able to get advice from current AI safety grantmakers. You might be able to fund work at the intersection of short timelines and the priorities of existing orgs, so there might be room for collaboration.

Imo, prizes/awards are less motivating, since they're not predictable enough to use to cover costs of living, so only people with other sources of income, financial support or savings can work full-time on prize problems.

Is this a good way to bet on short timelines?

I mostly agree with Rohin's answer, and I'm pretty skeptical overall of AI safety as a cause area, although I have deep uncertainty about this and might hedge by supporting s-risk-focused work.

Are you primarily interested in these trades with people who already prioritize AI safety?

On 3, do you mean you'd start putting x% after the first 5 years?

I think it's plausible you could find people who are undecided between AI safety with short timelines and other cause areas or between short and long timelines, and pay them enough to work on AI safety for short timelines, since they could address their uncertainty with donations outside of (short timeline) AI safety. I've worked as a deep learning researcher/engineer to earn-to-give for animal welfare, and I have considered working in AI safety, focusing on worst-case scenarios (CLR, CRS) or to earn-to-give for animal welfare. I think technical AI safety would be more interesting and motivating than my past work in deep learning, and perhaps more interesting day-to-day than my current plans but less motivating in the long run due to my skepticism. I was preparing to apply to CLR's internship for this past summer, but got an internship offer from Charity Entrepreneurship first and decided to go with that. I know one person who did something similar but went with AI safety instead.

It might be too expensive to pay people interested in earning-to-give enough to earn-to-give in (short timeline) AI safety, if AI safety isn't already one of their top priorities. Also, they don't even have to be EAs; you could find people who would just find the work interesting (e.g. people with graduate degrees in related subjects) but are worried about it not paying enough. You could take out loans to do this, but this kind of defies common sense and sounds pretty crazy to me.

(FWIW, my own price to work on AI safety (short or long timelines) is probably too high now, and, of course, there's the question of whether I'm a good fit, anyway.)

Load More