Harrison D

Wiki Contributions


‘High-hanging Fruits’ and Coordination

I think it’s good to consider these kinds of hypotheticals, and I don’t know if I’ve seen it done very often (although someone may have just expressed similar ideas in different terms). I’m also a big fan of recognizing stairstep functions and the possibility of (locally) increasing marginal utility—although I don’t think I’ve typically labeled those kinds of scenarios as “high-hanging fruits” (which is not to say the label is wrong, I just don’t recall ever thinking of it like that). That being said, I did have a couple thoughts/points of commentary:

• There are a variety of valid reasons to be skeptical of such “high-hanging fruit” and favorable towards “low-hanging fruit”—a point which I think other people would be more qualified to explain, but which relates to e.g., the difficulty of finding reliable evidence that massive increases in effort would have increasing marginal returns (since that may mean it hasn’t been tested at the theorized scale for impact yet), the heuristic of “if it’s so obvious and big , why are we only now hearing about it” (when applicable/within reason of course), the 80-20/power law distribution principle combined with the fact that a decent amount of low-hanging fruit still seems ripe for picking (e.g., low-cost, high-impact health interventions that still have room for funding), and so on.

• I suspect that as the benefits of/need for mass mobilization become fairly obvious, the general public would be more willing to mobilize, making the EA community less uniquely necessary (although this definitely depends on the situation in question).

• The EA community may not be sufficiently (large*coordinated) to overcome the marginal impact barriers.

• Applying the previous points to the scenarios you lay out (which I recognize are not the only possible examples, but it still helps to illustrate): for hypothetical 2, regarding climate change, this seems to be a case where it seems very unlikely that the general public will fail to support such a promising intervention while the EA community would be the unique tipping point to reach massive impact; in hypothetical 1, regarding animal advocacy, such a situation seems fairly unlikely when considering the reasoning behind diminishing marginal returns and historical experience with animal advocacy (e.g., there are going to be people who are difficult to convince), and again it seems unlikely (albeit less unlikely) that EAs would be the unique tipping point for massive impact.

• Especially considering the previous points, I don’t know how widespread/bad the assumptions of diminishing marginal returns in the EA community actually are in practice relative to its inaccuracy. I think it probably is an area for improvement/refinement, but it’s always possible to overcorrect, and I suspect that when/where increasing marginal returns becomes fairly obvious EAs would do a half-decent job of recognizing it (at least relative to how well they would recognize it if they thought about these scenarios more often).

Pascal's Mugging and abandoning credences

I do understand what you are saying, but my response (albeit as someone who is not steeped in longtermist/X-risk thought) would be "not necessarily (and almost certainly not entirely)."The tl;dr version is "there are lots of claims about X-risks and interventions to reduce x-risks that are reasonably more plausible than their reverse-claim." e.g., there are decent reasons to believe that certain forms of pandemic preparations reduce x-risk more than they increase x-risk. I can't (yet) give full, formalistic rules for how I apply the trimming heuristic, but some of the major points are discussed in the blocks below.

One key to using/understanding the trimming heuristic is that it is not meant to directly maximize the accuracy of your beliefs, rather it's meant to improve the effectiveness of your overall decision-making *in light of constraints on your time/cognitive resources. * If we had infinite time to evaluate everything--even possibilities that seem like red herrings--it would probably (usually) be optimal to do so, but we don't have infinite time so we have to make decisions as to what to spend our time analyzing and what to accept as "best-guesstimates" for particularly fuzzy questions. Here, intuition (including "when should we rely on various levels of intuition/analysis") can be far more effective than formalistic rules.

I think another key is to understand the distinction between risk and uncertainty: (to heavily simplify) risk refers to confidently verifiable/specific probabilities (e.g., a 1/20 chance of rolling a 1 on a standard 20-sided die) whereas uncertainty refers to when we don't confidently know the specific degree of risk (e.g., the chance of rolling a 1 on a confusingly-shaped 20-sided die which has never rolled a 1 yet, but perhaps might eventually).

In the end, I think my 3-4-ish conditions or at least factors for using the trimming heuristic are:

  1. There is a high degree of uncertainty associated with the claim (e.g., it is not a well-established fact that there is a +0.01% chance of extinction upon enacting this policy)

  2. The claim seems rather implausible/exaggerated on its face, but would require a non-trivial amount of time to clearly explain why (since it gets increasingly difficult to show why you ought to increase the number of zeros after a decimal point)

  3. You can quickly fight fire with fire (e.g., think of opposite-outcome claims like I described)

  4. There are other, more-realistic arguments to consider and your time is limited.

Pascal's Mugging and abandoning credences

In short:

  1. Bayesianism is largely about how to assign probabilities to things, it is not a ethical/normative doctrine like utilitarianism that tells you how you should prioritize your time. And as a (non-naïve) utilitarian will emphasize, when doing so-called “utilitarian calculus” (and related forms of analysis) is inefficient/less effective than using intuition, then you should rely on intuition.
  2. Especially when dealing with facially implausible/far-fetched claims about extremely high risk, I think it’s helpful to fight dubious fire with similarly dubious fire and then trim off the ashes: if someone says “there’s a slight (0.001%) chance that this (weird/dubious) intervention Y could prevent extinction, and that’s extremely important,” you might be able to argue that it is equally or even more likely that doing Y backfires or that doing Y prevents you from doing intervention Z which plausibly has a similar (unlikely) chance of preventing extinction. (See longer illustration block below)

In the end, these two points are not the only things to consider, but I think they tend to be the most neglected/overlooked whereas the complementary concepts are decently understood (although I might be forgetting something else).

Regarding 2 in more detail: Take for example classic Pascal's mugging-type situations, like "A strange-looking man in a suit walks up to you and says that he will warp up to his spaceship and detonate a super-mega nuke that will eradicate all life on earth if and only if you do not give him $50 (which you have in your wallet), but he will give you $3^^^3 tomorrow if and only if you give him $50." We could technically/formally suppose the chance he is being truthful is nonzero (e.g., 0.0000000001%), but still abide by rational expectation theory if you suppose that there are indistinguishably likely cases that cause the opposite expected value -- for example, the possibility that he is telling you the exact opposite of what he will do if you give him the money (for comparison, see the philosopher God response to Pascal's wager), or the possibility that the "true" mega-punisher/rewarder is actually just a block down the street and if you give your money to this random lunatic you won't have the $50 to give to the true one (for comparison, see the "other religions" response to the narrow/Christianity-specific Pascal's wager). More realistically, that $50 might be better donated to an X-risk charity. Add in the fact that stopping and thinking through this entire situation would be a waste of time that you could perhaps be using to help avert catastrophes in some other way (e.g., making money to donate to X-risk charities), and you’ve got a pretty strong case for not even entertaining the fantasy for a few seconds, and thus not getting paralyzed by naive application of expected value theory.

Why did EA organizations fail at fighting to prevent the COVID-19 pandemic?

the big EA organization that promise their donors to act against Global Catastrophic Risk were asleep and didn't react. [...]

Given that the got this so wrong, why should we believe that there other analysis of Global Catastrophic Risk isn't also extremely flawed?

Although I can't say I follow GCR's work, I'm unclear on whether/to what extent GCR or EAs in general necessarily "got this so wrong" or "were asleep and didn't react." I recognize it isn't always easy to demonstrate an absence/negative, but I really think you ought to provide some form of evidence or explanation for that claim--along with a reasonable standard for success vs. failure. To me, your post essentially can be summarized as "COVID-19 was due to lab leak; GCR/EAs are supposed to try to prevent pandemics, but a pandemic happened anyway, so how can we trust GCR/EAs?" Among other issues, this line of reasoning applies an excessively high standard for judging the efforts of GCR/EA: it's akin to saying "Doctors are supposed to help people, but some people in this hospital ended up dying anyway, so how can we trust doctors?"

Why scientific research is less effective in producing value than it could be: a mapping

Those both seem interesting! I'll definitely try to remember to reach out if I start doing more work in this field/on this project. Right now it's just a couple of ideas that keep nagging at me but I'm not exactly sure what to do with them and they aren't currently the focus of my research, but if I could see options for progress (or even just some kind of literature/discussion on the epistemap/repository concept, which I have not really found yet) I'd probably be interested.

Why scientific research is less effective in producing value than it could be: a mapping

Thanks for sharing those sources! I think a system like Pubpeer could partially address some of the issues/functions I mentioned, although I don't think it quite went as far as I was hoping (in part because it doesn't seem to have the "relies upon" aspect, but I also couldn't find that many criticisms/analyses in the fields I'm more familiar with so it is hard to tell what kinds of analysis takes place there). The Scite.ai system seems more interesting--in part because I have specifically thought that it would be interesting to see whether machine learning could assist with this kind of semantic-richer bibliometrics.

Also, I wouldn't judge based solely off of this, but the Nature article you linked has this quote regarding Scite's accuracy: "According to Nicholson, eight out of every ten papers flagged by the tool as supporting or contradicting a study are correctly categorized."

Why scientific research is less effective in producing value than it could be: a mapping

Yeah, I have thought that it would probably be nice to find a field where it would be valuable (based on how much the field is struggling with these issues X the importance of the research), but I've also wondered if it might be best to first look for a field that has a fitting/acceptive ethos--i.e., a field where a lot of researchers are open to trying the idea. (Of course, that would raise questions about whether it could see similar buy-in when applied to different fields, but the purpose of such an early test would be to identify "how useful is this when there is buy-in?")

At the same time, I have also recognized that it would probably be difficult... although I do wonder just how difficult it would be--or at least, why exactly it might be difficult. Especially if the problem is mainly about buy-in, I have thought that it would probably be helpful to look at similar movements like the shift towards peer-reviewing as well as the push for open data/data transparency: how did they convince journals/researchers to be more transparent and collaborative? If this system actually proved useful and feasible, I feel like it might have a decent chance of eventually getting traction (even if it may go slow).

The main concern I've had with the broader pipe dream I hinted at has been "who does the mapping/manages the systems?" Are the maps run by centralized authorities like journals or scientific associations (e.g., the APA), or is it mostly decentralized in that objects in the literature (individual studies, datasets, regressions, findings) have centrally-defined IDs but all of the connections (e.g., "X finding depends on Y dataset", "X finding conflicts with Z finding") are defined by packages/layers that researchers can contribute to and download from, like a library/buffet. (The latter option could allow "curation" by journals, scientific associations, or anyone else.) However, I think the narrower system I initially described would not suffer from this problem to the same extent--at least, the problems would not be more significant than those incurred with peer-review (since it is mainly just asking "1) does your research criticize another study? 2) What studies and datasets does your research rely on?")

But I would definitely be interested to hear you elaborate on potential problems you see with the system. I have been interested in a project of this sort for years: I even did a small project last year to try the literature mapping (which had mixed-positive results in that it seemed potentially feasible/useful but I couldn't find a great existing software platform to do both visually-nice mapping + rudimentary logic operations). I just can't shake the desire to continue trying this/looking for research or commentary on the idea, but so far I really haven't found all that much... which in some ways just makes me more interested in pursuing the idea (since that could suggest it's a neglected idea... although it could also suggest that it's been deemed impractical)

Why scientific research is less effective in producing value than it could be: a mapping

I was quite surprised/excited to see this on the forum, as I had literally just been thinking about it 5 minutes before opening up the forum!

From what I skimmed so far, I think it makes some good points/is a good start (although I'm not familiar enough in the area to judge); I definitely look forward to seeing some more work on this.

However, I was hoping to see some more discussion of the idea/problem that's been bugging me for a while, which builds on your second point: what happens when later studies criticize/fail to replicate earlier studies' findings? Is there any kind of repository/system where people can go to check if a given experiment/finding has received some form of published criticism? (i.e., one that is more efficient than scanning through every article that references the older study in the hopes of finding one that critically analyzes it).

I have been searching for such a system (in social sciences, not medicine/STEM) but thus far been unsuccessful--although I recognize that I may simply not know what to look for or may have overlooked it.

However, especially if such a system does not exist/has not been tried before, I would be really interested to get people's feedback on such an idea. I was particularly motivated to look into this because in the field I'm currently researching, I came across one experiment/study that had very strange results--and when I looked a bit deeper, it appeared that either the experiment was just very poorly set up (i.e., it had loopholes for gaming the system) or the researcher accidentally switched treatment group labels (based on internal labeling inconsistencies in an early version of the paper). As I came to see how the results may have just been produced by undergrad students gaming the real-money incentive system, I've had less confidence in the latter outcome, but especially if the latter case were true it would be shocking... and perhaps unnoticed.[1] Regardless of what the actual cause was, this paper was cited by over 70 articles; a handful explicitly said that the paper's findings led them to use one experimental method instead of another.

In light of this example (and other examples I've encountered in my time), I've thought it would be very beneficial to have some kind of system which tracks criticisms as well as dependency on/usage of earlier findings--beyond just "paper X is cited by paper Y" (which says nothing about whether paper Y cited paper X positively or negatively). By doing this, 1) there could be some academic standard that papers which emphasize or rely on (not just reference) earlier studies at least report whether those studies have received any criticism[2]; 2) If some study X reports that it relies on study/experiment Y, and study/experiment Y is later found to be flawed (e.g., methodological errors, flawed dataset, didn't reproduce), the repository system could automatically flag study X's findings as something appropriate like "needs review."

But I'm curious what other people think! (In particular, are there already alternatives/things that try to deal with this problem, is the underlying problem actually that significant/widespread, is such a system feasible, would it have enough buy-in and would it help that much even if it did have buy-in, etc.)

(More broadly, I've long had a pipe dream of some kind of system that allows detailed, collaborative literature mapping--an "Epistemap", if you will--but I think the system I describe above would be much less ambitious/more practical).

[1] This is a moderately-long digression/is not really all that important, but I can provide details if anyone is curious.

[2] Of course, especially in its early stage with limited adoption this system would be prone to false negatives (e.g., a criticism exists but is not listed in the repository), but aside from the "false sense of confidence" argument I don't see how this could make it worse than the status quo.

My attempt to think about AI timelines

Very small note: I'd recommend explaining your abbreviations at least once in the post (i.e., do the typical "full form (abbrev)"). I was already familiar with AGI, but it took me a few minutes of searches to figure out that TAI referred to transformative AI (no thanks to Tay, the bot).

Ending The War on Drugs - A New Cause For Effective Altruists?

[Incoming shameless self-promotion]

I think this is an example of where it may be helpful to move from the importance-tractability-neglectedness (INT) framework for selecting cause areas to looking more narrowly at the possible actions (or categories thereof, such as "voting; donating to political campaign groups; arguing for this in the public sphere/on social media; etc.") through the TUILS framework I've written about. The TUILS framework uses "trajectory/uniqueness" instead of "neglectedness", which means that it doesn't assume that the more neglected a cause/action is the better it is.

Load More