All of Harrison D's Comments + Replies

What is the likelihood that civilizational collapse would directly lead to human extinction (within decades)?

I was surprised to not see any mention of security dilemmas: one of the facially-plausible and otherwise commonly-mentioned scenarios/reasons I’ve seen for fighting or just wasting resources on combat preparations is a fear that “they might think that we think that they think… that someone wants to harm them, so we have to prepare/strike first.” In fact, security dilemmas are frequently suggested in conflict studies as reasons why two rational actors with 1) a preference for peace instead of violence, 2) realistically-bounded information about their advers... (read more)

An Open List of Legal Biosecurity Topics for Research/Law Students (US)

“There is nothing worse than duplicated/wasted efforts” - Definitely agree (at least in a sentimental rather than literal sense)

I know little to nothing about the legal-biosecurity nexus, and so I don’t know how interconnected or nuanced the topics tend to be. However, given my interests in improving research coordination, I’m curious whether people think shared mind maps and/or literature diagrams (e.g., a lite version of what I describe/illustrate here: (read more)

From the article: “I was convinced that this technology would eventually be everywhere, splashing techno-magic onto the world around us, impacting every domain from business and commerce to gaming and entertainment. Now, 30 years later, I am more convinced than ever that augmented reality will become central to all aspects of life, touching everything from how we work and play to how we communicate with each other. In fact, I am convinced that it will happen this decade — and yes, it will be magical.”

I would say it’s a fairly bold prediction, but then agai... (read more)

Why fun writing can save lives: the case for it being high impact to make EA writing entertaining

As I wrote in a separate comment, I did like this post overall. That being said, I would definitely push back on the idea that entertainment and truth are "unrelated." What you narrowly intended by that may or may not be true, but I can definitely say that there are many situations where truth requires nuance/complexity, and nuance/complexity can decrease entertainment and accessibility

On the one hand, there are many situations where people are already sufficiently familiar with the general ideas and/or those ideas are very easy to grasp such that w... (read more)

Why fun writing can save lives: the case for it being high impact to make EA writing entertaining

I'll agree that some people may still find "boring writing is unethical" to be off-putting, and I'll also say that I personally find it to be "odd." However, I definitely don't see it as off-putting as saying/implying "your career choice to be a [position that society tends to hold up as good] was immoral," for reasons I described above. My initial (and current) reading of your original comment (especially after seeing MichaelChen's response) was "I think this is odd and could be very off-putting; here's an analogy/example of why I think that." I would cer... (read more)

Why fun writing can save lives: the case for it being high impact to make EA writing entertaining

I feel a mixture of excitement and frustration in reading this: on the one hand, I have long been interested in communication ethics and analyzing/optimizing communication, and it's nice to see some of the ideas I've had relating to this getting both articulation and attention; on the other hand, I still don't see it going as deep or theoretical as what I'd like to see attempted

For example, it's nice to see a few numbers and individual concepts thrown around regarding impact estimates (e.g., slightly influencing a wide range of people vs. heavily in... (read more)

Why fun writing can save lives: the case for it being high impact to make EA writing entertaining

Personally, I share Ozzie’s concern about clickbait proliferation, but I didn’t think the title here was too bad: I think you can technically say (per utilitarianism) that sometimes being more boring is “unethical.” The point about ineffective giving and careers being off-putting is “correct,” but I don’t see that as really relevant to what Kat wrote here: those would be bad titles (in my view) primarily because they insult large personal choices that someone may have made in the past and which also tend to reflect/create a piece of someone’s identity (esp... (read more)

1Stefan_Schubert24dIt's just an example: I think it would also sound odd to say that minor decisions that reduce impact but are otherwise fine, e.g. from the perspective of common-sense morality, are "unethical".
Why Undergrads Should Take History Classes

Personally, I have tended to enjoy the history classes I took, but I also felt that actual history “classes” are not the most efficient way to learn broader lessons/concepts. For example, I took a Russian revolutionary (~1860–1921) history class, which I personally found interesting (especially since I was also studying Russian language). I feel like I learned quite a few interesting stories, and even learned some broader concepts about failure modes for democratic governance (e.g., division among more-moderate parties vs. cohesion among some of the more-m... (read more)

5ThomasWoodside1moClasses are often not the most efficient way to learn things. History is certainly no different, and I think the idea of history modules sounds very interesting. That being said, I wrote this post mainly for undergrads who have to operate within the boundaries of classes to some extent.
An estimate of the value of Metaculus questions

When I did some research on the use of forecasting to support government policymaking, one of the issues I quickly encountered was that for some questions, if your forecast is counterfactually (upon making the forecast) accurate and influential upon decision makers, it can lead to policies which prevent the event from occurring and thus making it an inaccurate forecast. Of course, some decisions are not about preventing some event from occurring but rather responding to such an event (e.g., preparedness for a hurricane), in which case there’s not much issu... (read more)

3NunoSempere1moMy thoughts are that this problem is, well, not exactly solved, but perhaps solved in practice if you have competent and aligned forecasters, because then you can ask conditional questions which don't resolve. * Given such and such measures, what will the spread of covid be. * Given the lack of such and such measures, what will the spread of covid be Then you can still get forecasts for both, even if you only expect the first to go through. This does require forecasters to give probabilities even when the question they are going to forecast on doesn't resolve. This is easier to do with EAs, because then you can just disambiguate the training and the deployment step for forecasters. That is, once you have an EA that is a trustworthy forecaster, you could in principle query them without paying that much attention to scoring rules.
Has anyone wrote something using moral cluelessness to "debunk" anti-consequentialist thought experiments?

Could you elaborate further what you have in mind with a moral cluelessness response?

Personally, I’ll say that most of those supposed anti-consequentialist thought experiments are helpful for illustrating why you need to be thoughtful when applying consequentialism (as is the case with every moral framework), but they do nothing to refute consequentialism. For example, there are many perfectly good utilitarian reasons to not conduct organ harvesting, such as threatening trust in institutions, risking your ability to do good in the future, the fact that you... (read more)

Should you optimise for the long-term survival of people who don't care for their own long-term survival?

It feels easier to do any form of moral advocacy on someone before they are in power, versus after they are in power.

This seems like a good point. And to your broader question, I do think it’s possible to get general rules of thumb (for personal though) and identify norms that ought to be social endorsed/pushed. However, you have to make sure to frame/ask the question correctly, which I think includes the point about “acceptable” vs. “beneficial” (and more generally, about taking utilitarianism as the foundational framework).

3Samuel Shadrach2moMakes sense. For personal, I can definitely see why acceptable and beneficial are different. I'm not sure how much the distinction matters for a society or hivemind. Whatever seems beneficial for society is what it should enforce norms towards and also deem acceptable. I feel like assuming utilitarianism will alienate people, might be better to keep the societal goal and corresponding norms more loosely and broadly defined. That way everyone individual can evaluate - does this society enforce social norms useful enough to my own personal goals both for myself and society - that I find it more value to accept and further enforce these social norms than rebel against them. Like how effective altruism forum doesn't explicitly refer to utilitarianism in the intro although the concepts are overlapping.
Nuclear Espionage and AI Governance

I’m surprised to not see much discussion about the highly dual-use and commercial-involvement aspects of AI in section 2 (unless I just missed it?). I have come to see that as one of the major differences between the Manhattan project/nuclear race and the hypothetical AI race. (Before I go into that point, I’ll just ask: did that come up in your research or did I just miss it?)

2GAA2moI agree this is a very important difference. I talk about it a little, but mostly not in section 2, which is more about categories of information (abstract vs. concrete vs. tacit, etc) than about institutional contexts for research. The most extensive discussion of issues of this kind is in section 3.4 where I basically say that in the private sector (where most AI development happens) financial motivations for spying are likely to be more important than they were in the Manhattan Project case because: 1. money is in general more salient as a motivation in the private sector 2. expected penalties are lower (being fired or sued rather than being imprisoned or executed) so it is easier to use money to outweigh the penalties and induce people to serve as spies There are a few other places where I touch on this issue as it relates to things like international trade in AI chips, or which countries will be favored by AI spying. But, aside from the stuff about financial motivations, it is treated in an ad hoc manner, in a sentence or two when it seems relevant to some other topic. If you have more general thoughts on the significance for AI espionage of the fact that AI has more important civilian applications than nuclear energy does, I would be very interested. Something that I researched a bit, but which did not make the final draft, is a case where the dual-use nature of nuclear energy enabled nuclear espionage. A. Q. Khan, a Pakistani engineer, worked at a Dutch civilian nuclear power plant. The plant did not have very stringent security because the Netherlands did not have nuclear weapons or particularly care about nuclear secrecy (at least at the time). Khan used what he learned in his role at the Dutch plant to serve as one of the most significant figures in the Pakistani nuclear weapons program (maybe the most significant, accounts differ). He went on to work as a sort of black market nuclear weapons consultant for (if I recall correctly) Iran, Syria, North Ko
Should you optimise for the long-term survival of people who don't care for their own long-term survival?

I was thinking of mandatory vaccinations, but I thought it seemed more like a case of requirements for community health rather than paternalism proper (focused on the wellbeing of the target of enforcement).

Should you optimise for the long-term survival of people who don't care for their own long-term survival?

I think it’s really tough to try to make some simplistic+universal set of rules. It also is really important to make sure you have a clear sense of what you are asking: “acceptable” (which is more in line with either a deontology framework or “would a utilitarian framework support social condemnation of this action [regardless of whether the action was itself utilitarian]”) may not be the same as “beneficial” (per a utilitarian framework). If it’s the latter, you might have rules of thumb but you will certainly struggle to get something comprehensive: the... (read more)

3Samuel Shadrach2moGreat answer and I tend to agree that a 100% comprehensive ruleset may be unobtainable. I wonder if we could still get meaningful rules of thumb even if not 100% comprehensive. And maybe these rules of thumb for what social norms are good can be generalisable across "whom" you're setting social norms or policy for. Maybe what social norms are good for "X choosing to respect or disrespect Y's autonomy" are similar whether: - X and Y are equal-standing members of the LW community - X is the parent of Y - X is a national law-making body and Y is citizens - X is programming the goals for an AGI that is likely to end up governing Y And as you mention, rules conditional on mental impairment or a sense of long-term wellbeing might end up on this list. Maybe I'll also explain my motivation in wanting to come up with such general rules even though it seems hard. I feel that we can't say for sure who will be in power (X) and who will be subjected to it (Y) in the future, but I do tend to feel power asymmetries will grow in the future. And there is some non-trivial probability that people from certain identifiable groups (scientists in certain fields, member of LW community, etc) end up in those positions of power. And therefore it might be worthwhile to cultivate those norms right here. It feels easier to do any form of moral advocacy on someone before they are in power, versus after they are in power. I understand if you still feel my approach to the problem is not a good one, I just wanted to share my motivation anyway.
Should you optimise for the long-term survival of people who don't care for their own long-term survival?

I want to be clear that there are certainly a lot of wrong ways to approach this, and that one should be very careful whenever they try to override or restrict someone’s decision-making: Generally, the instances where this violation of autonomy is clearly a good thing are quite rare in comparison to situations where it would be a bad thing. Also, I’ll clarify that “maximize their own preferences” probably wasn’t the best way of phrasing it (I typed that message in a rush while in transit)—a more accurate phrasing would have been something more like “maximi... (read more)

3Samuel Shadrach2moThanks for replying again. I'm just wondering if there's a way to condense down the set of rules or norms under which it is acceptable to take away someone's decision-making power. Or personally take decisions that will impact them but not respect their stated preferences. If I try rephrasing what you've said so far: 1. People with impaired mental capabilities Is it possible to universally define what classifies as mentally impaired here? Would someone with low IQ count? Someone with a brain disorder from birth? Someone under temporary psychedelic influence? Would an AI considering all humans stupid relative to its own intelligence count? 2. People whose actions or self-declared short-term preferences differ from _____ Should the blank be filled with "their self-declared long-term preferences" or "what you think their long-term preferences should be"? Or something else? I'm trying to understand what exactly wellbeing means here and who gets to define it.
Should you optimise for the long-term survival of people who don't care for their own long-term survival?

Second, briefly, I’ll add that “civilization” is not some monolithic moral patient, hive mind, or other thing: you can’t broadly say “civilization doesn’t want to be saved.” Regardless, I’ll bite the (nerf?) bullet here and bluntly say, I don’t really think it matters much if a hypothetical majority of the current ~7B people don’t want civilization to continue. It is important to consider why so many people might hypothetically think civilization shouldn’t continue (e.g., people have determined that the growth of civilization will produce net suffering ove... (read more)

3Samuel Shadrach2moThanks for the response. > they develop a level of self-rationality that typically does better than .... in maximizing their own preferences What does it mean for someone to undertake actions that are not maximising their own preferences? What does it mean to be rational when it comes to moral or personal values? Would I be right in assuming you're using a model where people have terminal goals which they can self-determine, but are then supposed to "rationally" act in favour of those terminal goals? And that if someone is not taking decisions that will take them closer to these goals (as decided by a rational mind), you feel it is morally acceptable (if not obligatory) that you take over their decision-making power?
Should you optimise for the long-term survival of people who don't care for their own long-term survival?

Two quick thoughts:

  1. I would definitely say there are good examples of so-called “paternalistic” policies: some people may engage in acts (e.g., suicide attempts) because they are suffering from temporary or long-term mental impairment. Additionally, I think nudge policies like opt-out instead of opt-in for saving money have generally been held up as good policy interventions. More broadly, I’d suggest there are many kinds of health and safety regulations which, although far from perfect as a whole, have probably as a whole helped people who would have wil
... (read more)
1Harrison D2moSecond, briefly, I’ll add that “civilization” is not some monolithic moral patient, hive mind, or other thing: you can’t broadly say “civilization doesn’t want to be saved.” Regardless, I’ll bite the (nerf?) bullet here and bluntly say, I don’t really think it matters much if a hypothetical majority of the current ~7B people don’t want civilization to continue. It is important to consider why so many people might hypothetically think civilization shouldn’t continue (e.g., people have determined that the growth of civilization will produce net suffering overall), but if they’re just being selfish (“I’d rather live a good life now than worry about future generations”) and in reality future civilization would produce significant wellbeing for however many billions or trillions of future people, then yeah, I wouldn’t worry too much about what current people think or want. Thankfully, I would argue we don’t live in such a world (even if people are too short-termist)
[PR FAQ] Tagging users in posts and comments

Seems like a fairly reasonable feature, and it’s quite common across many platforms.

EA Hangout Prisoners' Dilemma

This is certainly an interesting experiment to read about! Question though: Does it technically count as a true prisoner’s dilemma if the aggregate wellbeing increases with unilateral defection? I would have thought it wouldn’t be a prisoner’s dilemma unless the unilateral defection reward were something between $100 and $200, rather than $300…? If I really wanted $100 to be donated to one of the orgs but also liked the other one (albeit not to the same extent), I might have just said something like “I will commit to nuking one org, then I will verifiably commit to donating >$100 to the org that gets nuked.”

2Lukas_Finnveden2moAccording to wikipedia [], the $300 vs $100 is fine for a one-shot prisoner's dilemma. But an iterated prisoner's dilemma would require (defect against cooperate)+(cooperate against defect) < 2*(cooperate cooperate), since the best outcome is supposed to be permanent cooperate/cooperate rather than alternating cooperation/defection. However, the fact that this games gives out the same 0$ for both cooperate/defect and defect/defect means it nevertheless doesn't count as an ordinary prisoner's dilemma. Defecting against someone who defects needs to be strictly better than cooperating against a defector. In fact, in this case, every EA is likely going to put some positive valuation on $300 to both miri and amf, so cooperating against a defector is actively preferred to defecting against a defector.
5Jeffrey Ladish2moI expect most people to think either that AMF or MIRI is much more likely to do good. So from most agent's perspectives, the unilateral defection is only better if their chosen org wins. If someone has more of a portfolio approach that weights longtermist and global poverty efforts similarly, then your point holds. I expect that's a minority position though.
[Creative writing contest] Blue bird and black bird

Saw this quote somewhere and it made me think of this comment again: 

3WSCFriedman2moAh, but are there a thousand hacking at the branches of evil for each one who thinks they are striking at the root?
Is working on AI safety as dangerous as ignoring it?

Yeah, I haven't thought about this question previously and am not very familiar with AI safety research/debates (even though I occasionally skim stuff), but one objection that came to my mind when reading the original post/question was "If you aren't working on it, does that actually mean there will be one whole less person working on it?" Of course, I suppose it's possible that AI safety is somewhat weird/niche enough (in comparison to e.g., nursing, teaching) where the person-replacement ratio is moderate or low and/or the relative marginal returns of an... (read more)


Ah, I figured it was more of an argument against delayed giving rather than about plainly not giving. To clarify further, is your claim that the price of [saving?] a child’s life is actually doubling every few years (out of proportion to inflation), or is it just supposing a hypothetical world where that is the case?

1Rand3moI'm assuming it for the sake of the piece. I do think that the price of a child's life is rising faster than my investments appreciate, and probably thought they were doubling every 4 to 5 years when I wrote this. (I wrote $2000 back when I posted this to Facebook, I wonder what Givewell's estimates are.) (To clarify further, this was a post to my Facebook creative writing group in 2015 as was the "Responsibility" poetry I posted.)

Average annual stock market returns of roughly 9% while accounting for roughly 2% inflation, would see you double your money in about (70/(9%-2%)=10 years. It’s not exactly fast, but it’s doubling after accounting for inflation so it’s nothing to sneeze at. (Also worth noting, that’s not accounting for the additional deposits you make with your income, which would likely double a given amount much faster)

Of course, anyone should be careful about investing (e.g., potential for downturns or personal inability to make sound investment decisions in line with r... (read more)

1Rand3moThis piece isn't intended as an argument against delayed giving (though I think most such arguments would need to deny the premise of the piece). It's a story about not giving. It's about an older man, living in a time where saving a life in Kenya is like saving a life in Canada (that is, out of reach for most people), looking backward. Every year during that short window, he could have been a hero, saving one or more lives. He missed that chance and it doesn't exist anymore.
The Importance-Avoidance Effect

I think this post was written well and points out a potentially serious effect. I also think there is probably some validity to it—especially for some people it might be very strong—but I also like to wrinkle (or try to wrinkle) neat ideas, so I'll pose some potential objections I had while reading. In short, do you think it's possible there's a lot of correlation-causation confusion going on here? For example:

  1. Bigger and more-complex projects tend to be more important (or else we probably wouldn't do the project), and the complexity and size tend to make i
... (read more)
3davidhartsough3moThank you so much for your thoughtful response! I really appreciate your ideas. I'll reply inline here: Totally agree with this, and these things are compounding. Most of my claim is that some people struggle with a cognitive bias effect that pushes them to avoid all of this, all together. So you're right, it's not necessarily always the "importance" of the project that leads to avoidance or a struggle to effectively prioritize it; instead, it can be the complexity or a lack of clarity or vision that keeps the project out of arm's reach. Curiously, I often imagined that the people who struggle with the Importance-Avoidance Effect have typically already started their project(s) to some degree. And maybe they've even mapped out an outline of steps they need to follow to complete a series of milestones. Yet despite having had some start and having some clarity and vision, they lack the initiative and prioritization. This is very similar to what you describe as struggling to "get into a rhythm." I really like that phrasing. But anyways, there may be several compounding effects leading to a disruption in getting in rhythm. And it's not always just the "importance" alone that is to blame. But the fact is that sometime it is a major contributor, and for me, that has been something I've neglected to recognize for years! Hence why I want to bring attention to it. (We've known about our inabilities to tackle projects of great complexity. And now I'm hoping to expand on that understanding.) -------------------------------------------------------------------------------- That is fantastic that you are able to manage such dedication and prioritization. I'll be honest though, this is a major struggle for me and for some of my friends who I wrote this article about. For us, no matter how little skill is required and no matter how simple the task may be, the more important it becomes, the less it gets reasonably prioritized and worked on. (I have two ongoing projects that
Puggy's Shortform

I certainly would be interested in seeing such a system go into place—I think it would probably be beneficial—the main issue is just whether something like that is likely to happen. For example, it might be quite difficult to establish agreement between Charity Evaluator and GiveWell when it comes to the benefits of certain charities. Additionally, there may be a bit of survivor bias when it comes to organizations that have worked like FIDE, although I still think the main issue is 1) the analysis/measurement of effectiveness is difficult (requiring lots o... (read more)

[Creative writing contest] Blue bird and black bird

I did also initially think that it might be good to try to change the lumberjack instance, if possible, although it wasn't for the same reason: I just feel that there is much more of a case to make that the lumberjack deserves a whole-of-community effort since there is a plausible chance the extra bird could make a difference. But after considering this about the non-urgency of the sprout vs the lumberjack, I especially feel it may not the best example. Still, I understood the message/idea, and it's hard to know how non-EAs might react to the situation. Just something to keep in mind.

2Lizka3moThanks for the comments! The urgency argument makes sense. I'm not sure if I'll end up changing things, but I'll consider it, and thanks for pointing this out!
Puggy's Shortform

"But it’s got to confer status to the charity and people like Jay Z can gain more status by donating to it" - I think this brushes a good point which I'd like to see fleshed out more. On some level I'm still a bit skeptical in part because I think it's more difficult to make these kinds of designations/measurements for charities whereas things like album statuses are very objective (i.e., a specific number of purchases/downloads) and in some cases easier to measure. Additionally, for some of those cases there is a well-established and influential organizat... (read more)

1Puggy3moGreat points. Thank you for them. Perhaps we could use a DALY/QALY measure. A charity could reach the highest status if, after randomized controlled studies, it was determined that $10 donated could give one QALY to a human (I’m making up numbers). Any charity that reached this hard to achieve threshold would be given the super-charity designation. To make it official imagine that there’s a committee or governing body formed between charity navigator and GiveWell. 5 board members from each charity would come together and select the charities then announce the award once a year and the status would only be official for a certain amount of time or it could be removed if they dipped below a threshold. What do you think
[Creative writing contest] Blue bird and black bird

I can't say I've read that many of the creative fiction yet, but of those that I have read this is probably my favorite. It's simple, has nice/sweet illustrations, and not too heavy while also conveying some basic ideas. +2

2Lizka3moThanks a bunch--- I'm glad you liked it!

(Or maybe, "I invested the money and made above-inflation returns while also waiting to see further research on the most cost-effective interventions. I did occasionally make large donations with the investments when particularly opportune moments arose.")

2Rand3moIf you have a way of doubling your money every few years, go for it. But that's rather unlikely.
Magnitude of uncertainty with longtermism

I see what you mean, and again I have some sympathy for the argument that it's very difficult to be confident about a given probability distribution in terms of both positive and negative consequences. However, to summarize my concerns here, I still think that even if there is a large  amount of uncertainty, there is typically still reason to think that some things will have a positive expected value: preventing a given event (e.g., a global nuclear war) might have a ~0.001% of making existence worse in the long-term (possibility A), but it seems fair... (read more)

1Venky10243moThanks, that is fairly accurate summary of one of the crucial points I am making except I would also add that the difficulty of estimation increases with time. And this is a major concern here because the case of longtermism rests precisely on there being greater and greater number of humans (and other sentient independent agents) as the horizon of time expands. Fully agree that we should try but the case of longtermism remains rather weak until we have some estimates and bounds that can be reasonably justified.
Magnitude of uncertainty with longtermism

This seems to be an issue of only considering one side of the possibility distribution. I think it’s very arguable that a post-nuclear-holocaust society is just as if not more likely to be more racist/sexist, more violent or suspicious of others, more cruel to animals (if only because our progress in e.g., lab-grown meat will be undone), etc. in the long term. This is especially the case if history just keeps going through cycles of civilizational collapse and rebuilding—in which case we might have to suffer for hundreds of thousands of years (and subject ... (read more)

4Venky10243moYou're completely correct about a couple of things, and not only am I not disputing them, they are crucial to my argument: first, that I am only focusing on only one side of the distribution, and the second, that the scenarios I am referring to (with WW2 counterfactual or nuclear war) are improbable. Indeed, as I have said, even if the probability of the future scenarios I am positing is of the order of 0.00001 (which makes it improbable), that can hardly be the grounds to dismiss the argument in this context simply because longtermism appeals precisely to the immense consequences of events whose absolute probability is very low. At the risk of quoting out of context []: In much the same way, it's absolutely correct that I am referring to one side of the distribution ; however it is not because the other-side does not exist or is not relevant bur rather because I want to highlight the magnitude of uncertainty and how that expands with time. It follows also that I am in no way disputing (and my argument is somewhat orthogonal to) the different counterfactuals for WW2 you've outlined.
Magnitude of uncertainty with longtermism

(Note: I'm not well-steeped in the longtermism literature, so don't look to me as some philosophical ambassador; I'm only commenting since I hadn't seen any other answers yet.)

I get lost with your argument when you say "the standard deviation as a measure of uncertainty [...] could be so large that the coefficient of variation is very small" What is the significance/meaning of that?

I read your Medium post and I think I otherwise understand the general argument (and even share similar concerns at times). However,  my response to the argument you lay ou... (read more)

3Venky10243moThanks for the response. I believe I understand your objection but it would be helpful to distinguish the following two propositions: a. A catastrophic risk in the next few years is likely to be horrible for humanity over the next 500 years. b. A catastrophic risk in the next few years is likely to to leave humanity (and other sentient agents) worse off in the next 5,000,000 years, all things considered. I have no disagreement at all with the first but am deeply skeptical of the second. And that's where the divergence comes from. The example of a post-nuclear generation being animal-right sensitive is just one possibility that I advanced; one may consider other areas such as universal disarmament, open borders, end of racism/sexism. If the probability of a more tolerant humanity emerging from the ashes of a nuclear winter is even 0.00001, then from the perspective of someone looking back 100,000 years from now, it is not very obvious that the catastrophic risk was bad, all things considered. For example, whatever the horrors of WWII may have been, the sustenance of relative peace and prosperity of Europe since 1945 owes a significant deal to the war. In addition, the widespread acknowledgement of norms and conventions around torture and human rights is partly a consequence of the brutality of the war. That of course if far from enough to conclude that the war was a net positive. However 5000 years into the future, are you sure that in the majority of scenarios, in retrospect, WW2 would still be a net negative event? In any case, I have added this as well in the post:
A Primer on the Symmetry Theory of Valence

Haha, I certainly wouldn't label what you described/presented as "timecube weird." To be honest, I don't have a very clear cut set of criteria, and upon reflection it's probable that the prior is a bit over-influenced by my experiences with some social science research and theory as opposed to hard science research/theory. Additionally, it's not simply that I'm skeptical of whether the conclusion is true, but more generally my skepticism heuristics for research is about whether whatever is being presented is "A) novel/in contrast with existing theories or ... (read more)

3MikeJohnson3moHi Harrison, that’s very helpful. I think it’s a challenge to package fairly technical and novel research into something that’s both precise and intuitive. Definitely agree that “harmony” is an ambiguous concept. One of the interesting aspects of this work is it does directly touch on issues of metaphysics and ontology: what are the natural kinds of reality? What concepts ‘carve reality at the joints’? Most sorts of research can avoid dealing with these questions directly, and just speak about observables and predictions. But since part of what we’re doing is to establish valence as a phenomenological natural kind, we have to make certain moves, and these moves may raise certain yellow flags, as you note, since often when these moves are made there’s some philosophical shenanigans going on. That said, I’m happy with the overall direction of our work, which has been steadily more and more empirical. One takeaway that I do hope I can offer is the deeply philosophically unsatisfactory nature of existing answers in this space. Put simply, no one knows what pleasure and suffering are, or at least have definitions that are coherent across all domains they’d like to be able to define them. This is an increasing problem as we tackle e.g. problems of digital sentience and fundamental questions of AI alignment. I’m confident in our research program, but even more confident that the questions we’re trying to grapple with are important to try to address directly, and that there’s no good ’default hypothesis’ at present.
A Primer on the Symmetry Theory of Valence

I’m a bit hesitant to upvote this comment given how critical it is [was] + how little I know about the field (and thus whether the criticism is deserved), but I’m a bit relieved/interested to see I wasn’t the only one who thought it sounded really confusing/weird. I have somewhat skeptical priors towards big theories of consciousness and suffering (sort of/it’s complicated) + towards theories that rely on lots of complicated methods/jargon/theory (again, sort of/with caveats)—but I also know very little about this field and so I couldn’t really judge. Thus, I’m definitely interested to see the opinions of people with some experience in the field.

4MikeJohnson3moHi Harrison, appreciate the remarks. My response would be more-or-less an open-ended question: do you feel this is a valid scientific mystery? And, what do you feel an answer would/should look like? I.e., correct answers to long-unsolved mysteries might tend to be on the weird side, but there’s “useful generative clever weird” and “bad wrong crazy timecube weird”. How would you tell the difference?
The expected value of funding anti-aging research has probably dropped significantly

Just to be clear: is your point that because the industry is already/now getting lots of money from billionaires the marginal value of donating additional money is smaller? And/or is it (also) that donating will lead to billionaires donating less?

1freedomandutility3moThe first! (And not the second). I’m not 100% sure if ‘subsidising billionaires’ is the correct term but I mean that money donated towards aging is probably going to be donated by billionaires anyway.
Neglected biodiversity protection by EA.

I was hoping to see some more answers by now, but seeing none I'll provide some initial points that I expect people who are more familiar with the field could flesh out much better than I. I'm not familiar with the full case for biodiversity, but just based on generic reasoning about cause areas in general, I'd suspect some of the major reasons EAs do not seem to consider it as important as factory farming, wild animal welfare, or even climate change (to name a few environmentally-oriented cause areas) include:

  1. It's unclear how significant the extrinsic, we
... (read more)
Neglected biodiversity protection by EA.

I'm still not a huge fan of the way it is written—it sounds almost like a strawman description of wild animal welfare work. In particular, I don't think adding "maybe" does enough blunt/caveat the second part of the sentence, which is not presented very delicately.

How to get more academics enthusiastic about doing AI Safety research?

I’m not an academic nor do I otherwise have much familiarity with academia careers, but I have occasionally heard people talk about the importance of incentive structures like tenure publication qualification, the ease of staying in established fields, the difficulty/opportunity cost of changing research fields later in your career, etc. Thus I think it would be helpful/interesting to look at things more from that incentive structure side of things in addition to asking “how can we convince people AI safety is important and interesting?”

3PabloAMC3moI agree that the creation of incentives is a good framing for the problem. I wanted to notice some things though: * Academics often have much more freedom to research what they want, and most incentives are number of publications or citations. Since you can publish AIS papers in standard top conferences, I do not see a big problem, although I might be wrong, of course. * Changing the incentives is either more difficult (changing protocols at universities or government bodies?) or just giving money, which the community seems to be doing already. That's what makes me think that the academic interest is more of a bottleneck, but I am not superinformed.
When pooling forecasts, use the geometric mean of odds

Those equations are definitely less intimidating / I could understand them without an issue. (And that’s totally fine, I hope it’s helpful)

When pooling forecasts, use the geometric mean of odds

(Am I the only one who wasn’t familiar with geometric means and thought the explanation/equation was intimidating? (Edit/note: this no longer applies after the OP's update))

Unless I missed it somewhere, I would definitely recommend explaining geometric vs. arithmetic mean in simple English terms early on. Not explaining mathematical equations in lay terms is a minor a pet peeve of mine; I feel that in this case it could have been explained better for non-mathematical audiences by simply saying “Whereas the arithmetic mean adds the values together and divides by the number of values, the geometric mean multiplies all the values and then takes the n-th root of the product (where n = number of values).”

8Jsevillamol3moYou are right and I should be more mindful of this. I have reformulated the main equations using only commonly known symbols, moved the equations that were not critical for the text to a footnote and added plain language explanations to the rest. (I hope it is okay that I stole your explanation of the geometric mean!)
Frank Feedback Given To Very Junior Researchers

“ I personally find it fairly irritating/annoying.”

What may be true for you may not be true for others. And all of us like to think we are capable of dispassionately analyzing and evaluating feedback, but sometimes monkey brain + emotions + deeply negative feedback = error

Promoting Simple Altruism

Since I don’t see it mentioned already, one of the major “cause areas” analyzed by e.g., 80000 Hours is “meta-EA” (promoting effective altruism). I understand that one of your points is about trying to promote altruism more generally, and I too have wondered about the potential benefits/tradeoffs of watering down the EA message to try to reach more people (e.g., encouraging people to at least donate to decently effective charities—perhaps “try to find the most effective charity for the problem you want to focus on [even if the overall problem area isn’t re... (read more)

2LiaH3moYes, these are all sound counterpoints. Together, they suggest the idea is at least, neglected. I think your point 2 was also made by Stefan_Schubert in a comment above. I would be very interested to see research in the area, if there is any. I agree your points 1&3 are a problem if the number of altruistic people were finite, but what if everyone behaved altruistically, to the benefit of others? To the point that it would not matter if some people chose to donate to seeing eye dog charities? I can appreciate your argument that promoting general altruism might not fit under EA banner, specifically because it lacks the "effective" intent, but I would argue that it could be one of the "hits based", fat-tailed investments EA has been seeking. What if it were tractable and scalable to make people generally nicer to each other, and desire to help each other, non-human animals, the environment, and the future, impartially?
"Epistemaps" for AI Debates? (or for other issues)

Yeah, I definitely felt that one of the downsides of Semantica Pro (or at least, my version of it) was the lack of quantitative or even logic (if-then) functionality, which in my mind would be a crucial feature. For example, I would want to see some kind of logical system that flags claims that depend on an assumption/claim/study that is shown to be flawed (and thus the flagged claims may need to be reevaluated). In my most recent research project, for example, I found a study which used a (seemingly/arguably) flawed experiment design for testing predictio... (read more)

Frank Feedback Given To Very Junior Researchers

I had similar thoughts at the end. I definitely think the optimal feedback depends on the audience/recipient and the quality of the project (including whether the person seems to have an accurate vs. overly-pessimistic vs. overly-optimistic view of the project quality), but I also think that in most cases it's probably better to add more positive notes, especially at the beginning and end.

The Scientist Mode, the Scout Mindset and Flashcards

I'm not really sure why this appears to have been downvoted; I only skimmed the article so IDK if there was something really controversial or wrong that I missed, but it might just be that the information here could probably be summarized more effectively at the beginning (or at the end, at least). I have a decent understanding of the "scientist mode" in part from the name alone, and I've read Scout Mindset; I think personally I'd want to just read a summary list of "where are these two mindsets different? Why might I want to choose one over the other?" Otherwise, I think this is a fine summary of the points (based on the parts I did read)

Are many EAs philosophical pragmatists?

Could you clarify if you have in mind any other characteristics that determine whether someone is or isn't a philosophical pragmatist aside from "[believing] that we can reason our way toward capital-T Truth"? Although I've encountered discussions of philosophical pragmatism before, it's definitely not my area of expertise. Additionally, I've not had this conversation about philosophical pragmatism with many EAs. 

That being said, I think it's reasonable to say that many if not most EAs probably believe "objectively verifiable 'Truth' is not attainable... (read more)

3taoroalin@gmail.com3moThere are also like 3 different ways 2+2!=4. Outer universe with different math - We're a simulation inside a different universe that runs on different math where 2+2!=4, but the math inside our universe is consistent. This is the same as 2+2=4 for most purposes. This is imaginable, I think... Active demon - there's a demon that controls all your inputs, in a way that's inconsistent with any reasonable mathematics, but you can't tell. This is the least likely, and if it were true I wouldn't even consider myself a person. Math is flawed - the whole concept of arithmetic, or all of mathematics, is inconsistent and it's impossible to construct a system where you can actually prove 2+2=4. This doesn't necessarily mean two apples and two apples makes four apples - it just means the apples behave like they do for other reasons than arithmetic. This is conceivable.
5rorty3moThanks for this. Here's Stanford Encyclopedia of Philosophy's first paragraph: "Pragmatism is a philosophical tradition that – very broadly – understands knowing the world as inseparable from agency within it. This general idea has attracted a remarkably rich and at times contrary range of interpretations, including: that all philosophical concepts should be tested via scientific experimentation, that a claim is true if and only if it is useful (relatedly: if a philosophical theory does not contribute directly to social progress then it is not worth much), that experience consists in transacting with rather than representing nature, that articulate language rests on a deep bed of shared human practices that can never be fully ‘made explicit’." I'd say pragmatism is as much a criticism of certain directions in philosophy as anything. It's a method of asking of philosophical distinctions "what difference would that make." But instead of turning toward skepticism it seeks to reorient philosophy and reasoning toward the fulfillment of human goals.
"Epistemaps" for AI Debates? (or for other issues)

I'm glad to hear you are interested, and I appreciate the resources/links!

Re (1): I'm a big fan of Kialo (even though I have not been active in community discussions in the past few years). I actually have been working a forum post that would highlight its potential use for some issues in the EA community.  Still, Kialo is much more narrowly focused on pro/con argumentation structure rather than diagramming concepts/observations/etc. more broadly. Additionally, I have seen Bayesian networks (although I can't remember if I saw Norsys), but these tend t... (read more)

1Paal Fredrik Skjørten Kvarberg3moGood! Yeah, I didn't mean to say that any of these capture all the specifics of your idea, but merely that there is a lot of interest in this sort of thing. It's probably worthwhile pursuing this in more detail, I'd be interested in seeing more on this.
"Epistemaps" for AI Debates? (or for other issues)

What you describe there is probably one of the most similar concepts I've seen thus far, but I think a potentially important difference is that I am particularly interested in a system that allows/emphasizes semantically-richer relationships between concepts and things. From what I saw in that post, it looks like the relationships in the project you describe are largely just "X influences Y" or "X relates to/informs us about  Y", whereas the system I have in mind would allow identifying relationships like "X and Y are inconsistent claims," "Z study ha... (read more)

4Davidmanheim3mo(I'm also working on the project.) We definitely like the idea of doing semantically richer representation, but there are several components of the debate that seem much less related to arguments, and more related to prediction - but they are interrelated. For example, Argument 1: Analogies to the brain predict that we have sufficient computation to run an AI already Argument 2: Training AI systems (or at least hyperparameter search) is more akin to evolving the brain than to running it. (contra 1) Argument 2a: The compute needed to do this is 30 years away. Argument 2b (contra 2a): Optimizing directly for our goal will be more efficient. Argument 2c (contra 2b): We don't know what we are optimizing for, exactly. Argument 2d (supporting 2b): We still manage to do things like computer vision. Each of these has implications about timelines until AI - we don't just want to look at strength of the arguments, we also want to look at the actual implication for timelines. Semantica Pro doesn't do quantitative relationships which allow for simulation of outcomes and uncertainty, like "argument X predicts progress will be normal(50%, 5%) faster." On the other hand, Analytica doesn't really do the other half of representing conflicting models - but we're not wedded to it as the only way to do anything, and something like what you suggest is definitely valuable. (But if we didn't pick something, we could spend the entire time until ASI debating preliminaries or building something perfect for what we want,) It seems like what we should do is have different parts of the issue represented different / multiple ways, and given that we've been working on cataloging the questions, we'd potentially be interested in collaborating.
Harrison D's Shortform

(Summary: A debate league's yearlong policy debate resolution is about AI; does this seem like a good outreach opportunity?)

"Resolved: The United States Federal Government should substantially reform the use of Artificial Intelligence technology."

IMO, it's not the best of wording, but that's the current team policy debate resolution in the Stoa debate league. For the next ~9 months, a few hundred high school students will be researching and debating over "the use of artificial intelligence technology." In the past, people have posted about competitive deba... (read more)

3Larks3moGreat spot. Presumably this means a lot of kids will be googling related terms and looking for pre-existing policy suggestions and pro/con lists.
Can you control the past?

I’ve seen similar discussions of EDT vs. CDT and most of the associated thought experiments elsewhere, but the emphasis here seems to be much more about whether you can actually have a causal impact on the past. You’ll have to forgive me if you address this set of points+objections somewhere and I just missed it (it’s a long post!), but my thought process is:

  1. Most (if not all) of the situations you describe seem to assume away important beliefs about what is physically/epistemically possible in terms of predictive accuracy. This seems to contribute to a l
... (read more)
2Joe_Carlsmith3mo"the emphasis here seems to be much more about whether you can actually have a causal impact on the past" -- I definitely didn't mean to imply that you could have a causal impact on the past. The key point is that the type of control in question is acausal. I agree that many of these cases involve unrealistic assumptions, and that CDT may well be an effective heuristic most of the time (indeed, I expect that it is). I don't feel especially hung up on calling it "control" -- ultimately it's the decision theory (e.g., rejecting CDT) that I'm interested in. I like the word "control," though, because I think there is a very real sense in which you get to choose what your copy writes on his whiteboard, and that this is pretty weird; and because, more broadly, one of the main objections to non-CDT decision theories is that it feels like they are trying to "control" the past in some sense (and I'm saying: this is OK). Simulation stuff does seem like it could be one in principle application here, e.g.: "if we create civilizations simulations, then this makes it more likely that others whose actions are correlated with ours create simulations, in which case we're more likely to be in a simulation, so because we don't want to be in a simulation, this is a reason to not create simulations." But it seems there are various empirical assumptions about the correlations at stake here, and I haven't thought about cases like this much (and simulation stuff gets gnarly fast, even without bringing weird decision-theory in).
Load More