All of jackva's Comments + Replies

My sense is that it is not a big priority.

However, I would also caution against the view that expected climate risk has increased over the past years.

Even if impacts are faster than predicted, most GCR-climate risk does probably not come from developments in the 2020s, but on emissions paths over this century.

And the big story there is that the expected cumulative emissions have much decreased (see e.g. here).

As far as I know no one has done the math on this, but I would expect that the decrease in likelihood of high warming futures dominates somewhat high... (read more)

Even if one is skeptical of the detailed numbers of a cost effectiveness analysis like this (as I am), I think it is nonetheless pretty clear that this 1M spent was a pretty great bet:

1. When I talked to ITIF in 2020, they were pretty clear how transformative the Let's Funds campaign had been for their fundraising. 
2. Given the amount of innovation-related decision making that occurred in the run-up to and early Biden administration -- what became the IIJA, CHIPS, and IRA, probably the largest expansion of energy innovation activity in decades -- significan

"Pyramid scheme" has a new meaning.

I am also just beginning to think about this more, but some initial thoughts:

• Path dependency from self-ampliying processes -- Thinking about model generations as forks where significant changes in the trajectory become possible (e.g. crowding in a lot more investment, as has happened with ChatGPT/GPT4, but also, as has also happened, a changed Overton window). I think overall this introduces a dynamic where the extremes of the scenario space become more likely, with social dynamics such as strong increase in investment or, on the other side, stricter regul

I agree with you that the 2018 report should not have been used as primary evidence for CATF cost-effectiveness for WWOTF (and, IIRC, I advised against it and recommended an argument more based on landdscaping considerations with leverage from advocacy and induced technological change). But this comment is quite misleading with regards to FP's work as we have discussed before:

1. I am not quite sure what is meant with "referencing it", but this comment from 2022 in response to one of your earlier claims already discusses that we (FP) have not been using that e

It seems like that this number will increase by 50% once FLI (Foundation) fully comes online as a grantmaker (assuming they spend 10%/year of their USD 500M+ gift)

https://www.politico.com/news/2024/03/25/a-665m-crypto-war-chest-roils-ai-safety-fight-00148621

Interesting, thanks for clarifying!

Just to fully understand -- where does that intuition come from? Is it that there is a common structure to high impact? (e.g. if you think APs are good for animals you also think they might be good for climate, because some of the goodness comes from the evidence of modular scalable technologies getting cheap and gaining market share?)

I don't think these examples illustrate that "bewaring of suspicious convergence" is wrong.

For the two examples I can evaluate (the climate ones), there are co-benefits, but there isn't full convergence with regards to optimality.

On air pollution, the most effective intervention for climate are not the most effective intervention for air pollution even though decarbonization is good for both.
See e.g. here (where the best intervention for air pollution would be one that has low climate benefits, reducing sulfur in diesel; and I think if that chart were... (read more)

Fascinating stuff!

I am curious how you think about integrating social and political feedback loops into timeline forecasts.

Roughly speaking, (a) when we remain in the paradigm of relatively predictable progress (in terms of amount of progress, not specific capabilities) enabled by scaling laws, (b) we put significant probability on being fairly close to TAI, e.g. within 10 years, (c) it remains true that model progress is clearly observable by the broader public,

then it seems that social and political factors might drive a large degree in the variance of e... (read more)

but that a priori we should assume diminishing returns in the overall spending, otherwise the government would fund the philanthropic interventions.

I think this is fundamentally the crux -- many of the most valuable philanthropic actions in domains with large government spending will likely be about challenging / advising / informationally lobbying the government in a way that governments cannot self-fund.

Indeed, when additional government funding does not reduce risk (does not reduce the importance of the problem) but is affectable, there can probably be cases where you should get more excited about philanthropic funding to leverage as public funding increases.

Yeah, that's true, though in Luke's treatment both are discussed and described as roughly equal -- there's no indication given that either should be more promising on priors and, as you say, they will often overlap.

(Last comment from me on this for time reasons)

• I think if you look at philanthropic neglectedness, the total sums across types of capital are not a good proxy. E.g., as far as I understand the nuclear risk landscape, it is both true that government spending is quite large but also that there is almost no civil society spending. This means that additional philanthropic funding should be expected to be quite effective on neglectedness grounds. Many obvious things are not done.
• The numbers on nuclear risk spending by 80k are entirely made up and not described

I can't open the GDoc on AI safety research.

But, in any case, I do not think this works, because philanthropic, private, and government dollars are not fungible, as all groups have different advantages and things they can and cannot do.

If looking at all resources, then 80M for AI safety research also seems an underestimate as this presumably does not include the safety and alignment work at companies?
 

Nuclear risk philanthropy is about 30M/y, it seems you are comparing overall nuclear risk effort to philanthropic effort for AI?

In terms of philanthropic effort AI risk strongly dominates nuclear risk reduction.

Sorry for not being super clear in my comment, it was hastily written. Let me try to correct:

I agree with your point that we might not need to invest in govt "do something" under your assumptions (your (1)).

I think the point I disagree with is the implicit suggestion that we are doing much of what would be covered by (1). I think your view is already the default view. 

• In my perception, when I look at what we as a community are funding and staffing, > 90% of this is only about (2) -- think tanks and other Beltway type work that is focused on make ac

Thanks, Jamie! Indeed quite helpful to know that there's nothing obvious I am missing.

Yes, agree on the last point -- I am just surprised this has not been done as EA grant makers frequently face the decision, I think.

Is there a process for more time-sensitive grants (where a decision would be needed earlier)?

This seems right to me on labs (conditional on your view being correct), but I am wondering about the government piece -- it is clear and unavoidable that government will intervene (indeed, already is) and that AI policy will emerge as a field between now and 2030 and that decisions early on likely have long-lasting effects. So wouldn't it be extremely important also on your view to now affect how government acts?

Thanks, good shout!

From what I've seen, their work does not quite fit what I am looking for -- they are not comparative and they are also more narrowly focused on left-leaning protest movements, which is more narrow than what I am trying to get at here.

I think it's useful to add some quantitative intuitions here:

Quick BOTEC:

• In the US there's a 50% chance a candidate not committed to democracy will become President, in Germany a party not being committed to democracy is part of the next government is very low, certainly not more than 0.5% (>=100x difference). There are certainly other ways to think about this, but I think the basic captured intuition - despite all the turmoil the threat to German democracy is low, in absolute terms and comparatively - seems correct (and, indeed, the recent protests see

In my experience, many of those arguments are bad and not cause-neutral, though to me your take seems too negative -- cause prioritization is ultimately a social enterprise and the community can easily vet and detect bad cases, and having proposals for new causes to vet seems quite important (i.e. the Popperian insight, individuals do not need to be unbiased, unbiasedness/intersubjectivity comes from open debate).

How do you think about the relevance of evidence from pre-1800/pre-industrialized societies to questions of whether climate change will induce civilizational collapse going forward?

To me, I am always confused why people do these studies because society has clearly changed so dramatically that there seems to be very little to learn from how these societies responded to climate anomalies.

I don't think IPBES is relevant evidence here because ~no one in the US cares about biodiversity as a national policy issue. It has no salience whatsoever, it is not something that can be polarized.

Founders Pledge

I agree on that. My point is more forward-looking and in terms of counterfactuals: when there is an opportunity to shape an issue now making it institutionally look more like climate with an IPCC-equivalent is risky given the political environment now.

That's why I specified "inside-climate", yes those considerations you mention are out of scope for stuff I can fund.

This is an aside, but I would not trust CCC on climate.

Thanks, Danny!

I think this is a misunderstanding.

I am not saying the IPCC caused polarization by something they did but rather by what they represent:

1. The IPCC and similar-style organizations can be used by rising populist anti-globalist, referring to an international scientific body as a reason to do something seems politically very risky when the reputation of science and of international institutions is lower than it used to be and a significant part of the electorate actively resents those authorities.

2. Insofar as the constraint on GCR risk-reductio

Are you interested in within-climate cruxes?

Interesting idea, thanks for writing this!

How do you think about the risk of this kind of move, modeled loosely after the IPCC for climate change? In particular, that it will make GCR mitigation more like climate change politically?

1. The IPCC emerged / became strong (obviously IPCC was founded in 1988 before the end of the Cold War, but most of its success came after) at a time where there was a lot of appetite for global cooperation and scientific input in environmental policy-making and, despite that, it failed to meaningfully shape the trajectory of cl... (read more)

Thanks for the update, Will!

As you are framing the choice between work on alignment and work on grand challenges/non-alignment work needed under transformative AI, I am curious how you think about pause efforts as a third class of work. Is this something you have thoughts on?

We agree on economics, it's more that techno-economic analysis is quite different (just had someone on my team do techno-economic work that would be relevant to this list, but she is maximally far from a social scientist in skillset and self-identification).

I think for some parts of social psychology it might be considered a social science, though in general most social scientists would say the definition of social science is something like "the dependent variable are societal-level phenomena" by which economics, political science, sociology etc. are socia... (read more)

I love that you are doing this!

I think a broader title might be more helpful, though -- many of the questions you list are not really social science questions, but, for example, about consumer psychology (a behavioral science) or techno-economics (e.g. the alt protein R&D return questions).

I.e. there is a broader set of people who might help answer these questions, many of which would not understand themselves as social scientists.

(I am pretty unsure I understood this correctly, so this comment might be a mistake, posting anyway as it might be clarifying for others as well if so)

It seems to me that there are two dimensions here:

(a) whether or not a statement is comparative (b) whether or not a statement is confounded by an unobservable

Comparative statements can be confounded when the comparison standard is not made explict, which seems to be your main critique. If I understand you correctly, you see the main response in non-comparative first order evaluations.

But shouldn't, in many ... (read more)

Thanks, and sorry if I was too nitpicky then.

I am not an GHD expert but I would expect someone who has a high school diploma in the richest country in Africa to be a lot better off than the typical GD recipient which seems to be from the poorest strata of the poorest countries.

And so, yeah, I agree one would probably a 50-100x expected multiplier to make this work. I am not saying this is not possible, I just thought the bar stated here was significantly too optimistic.

I suspect that it shouldn't be too hard to find one where spending $1 generates more than $10 in income, which is roughly the bar for a GiveWell top charity.

This seems wrong to me in that both of your examples are constituencies that are quite a bit better off than Give Directly recipients for which that would hold, i.e. the actual multiplier would need to be a lot higher or apply to constituencies as poor as GD-recipients. 

Hi Vasco,

Thanks for your thoughtful comment!

It took me a while to fully parse, but here are my thoughts, let me know if I misunderstood something.

I/ Re the 3000x example, I think I wasn't particularly clear in the talk and this is a misunderstanding resulting from that. You're right to point out that the expected uncertainty is not 3000x.

I meant this more to quickly demonstrate that if you put a couple of uncertainties together it quickly becomes quite hard to evaluate whether something meets a given bar, the range of outcomes is extremely large (if on reg... (read more)

Thanks, Luke!

Uncertainty
As we frequently point out, one should take the estimates with a grain of salt and consider the reported uncertainty (e.g. the old estimate had something like 0.1 USD/tCO2e to 10 USD/tCO2e) and, IIRC, the impact report also reports that these estimates are extremely uncertain and reports wide ranges.

As we discussed in our recent methodology-focused update, we think large uncertainty is unavoidable when operating in climate as a global decadal challenge with the most effective interventions inherently non-RCT-able (FWIW, I would thin... (read more)

It seems good to me if the forum team took more action here against this post, for example removing the section on Ben Pace that can clearly be interpreted as retaliatory. I don't see why we would assume good faith for that part of the post.

The reaction here of the moderation seems a bit unbalanced.

Thanks, Vasco, for the great comment, upvoted! I am traveling for work right now, but we'll try to get back to you by ~mid-week.

AI might kill us all but in the meantime there will be some great summaries.

Thanks for doing this!

As a climate person trying to have a balanced perspective on this, to me the framing of climate here does not come across as very balanced. @John G. Halstead might have more detailed comments on this, but it seems that examples are selectively chosen in one direction (motivating the severity of the risk).

I am extremely pro alternative proteins (see e.g. here) but I think we still need to be more honest about the climate impacts of agriculture, both in terms of epistemic hygiene but also in terms of argumentative strategy (I don’t think we need to exaggerate the case for APs – the case is good already! – and by exaggerating some claims we are making the whole thing less believable).

In the beginning of the interview it is discussed as a huge, huge contributor to climate change, a major driver, without presenting any numbers.

The exact numbers would depen... (read more)

Thanks so much! I know the problem of late answers :)

I think even for something that seems quite certain on the intervention level (if you think that is true for malaria vaccine) then one needs to account for funding and activity additionality which make this more uncertain and, relatively speaking, lowers the estimate to GD where the large size of the funding gap ensures funding and activity additionality to be near 1 (i.e. no discount).

Given that Open Philanthropy seems to believe that typical GiveWell recommendations are dominated by more leveraged ones (e.g.using advocacy, induced technological change) at least for risk-neutral donors, I am a bit confused by the anchoring on GiveWell charities.

Even if GD were closer to AMF than GiveWell thinks, this would not put GD close to the best thing one can do to improve human welfare unless one applies a very narrow frame (risk-aversion, highly scalable based on existing charities right now).

Or, put a bit differently:

• (1) We live in a world that

Thanks for doing this and kudos for publishing results that are in tension with your (occasional) employer.

Interesting to see a clear statement by OP on the expected dominance of advocacy and other leveraged interventions over traditional direct delivery work.

(Full disclosure: I sometimes work out of the same coworking space as Justus and Vegard and we occassionally have team lunches. Given that they were a potential grantee for some time (and indeed became a grantee for a small grant in 2023), I've avoided further socializing beyond those office contexts. They also don't know I am writing this.)

This is an exciting broadening of work!

I haven't always agreed with the underlying theory of change of the climate work, but I've consistently experienced the team of Future Matters as quite thoughtful about policy change and social movements and cultivating an expertise that is quite rare in EA and seems underprovided.

I think the idea of an energy descent is extremely far outside the expert consensus on the topic, as Robin discusses at length in his replies to that post.

This is nothing we need to worry about.