All of Linch's Comments + Replies

This seems like a pretty unlikely fallacy, but I agree it's theoretically possible (and ocassionally happens in practice).

The difference between 0 and 1 is significant! And it's very valuable to figure out when the transition point happens, if you can.

PSA: regression to the mean/mean reversion is a statistical artifact, not a causal mechanism.

So mean regression says that children of tall parents are likely to be shorter than their parents, but it also says parents of tall children are likely to be shorter than their children.

Put in a different way, mean regression goes in both directions. 

This is well-understood enough here in principle, but imo enough people get this wrong in practice that the PSA is worthwhile nonetheless.

I think something a lot of people miss about the “short-term chartist position” (these trends have continued until time t, so I should expect it to continue to time t+1) for an exponential that’s actually a sigmoid is that if you keep holding it, you’ll eventually be wrong exactly once.

Whereas if someone is “short-term chartist hater” (these trends always break, so I predict it’s going to break at time t+1) for an exponential that’s actually a sigmoid is that if you keep holding it, you’ll eventually be correct exactly once.

Now of course most chartists (my... (read more)

Also seems a bit misleading to count something like "one afternoon in Vietnam" or "first day at a new job" as a single data point when it's hundreds of them bundled together?

From a information-theoretic perspective, people almost never refer to a single data point as strictly as just one bit, so whether you are counting only one float in a database or a whole row in a structured database, or also a whole conversation, we're sort of negotiating price. 

I think the "alien seeing a car" makes the case somewhat clearer. If you already have a deep model of ... (read more)

EDIT: I noticed that in my examples I primed Claude a little, and when unprimed Claude does not reliably (or usually) get to the answer. However Claude 4.xs are still noticeable in how little handholding they need for this class of conceptual errors, Geminis often takes like 5 hints where Claude usually gets it with one. And my impression was that Claude 3.xs were kinda hopeless (they often don't get it even with short explanations by me, and when they do, I'm not confident they actually got it vs just wanted to agree).

"Most people make the mistake of generalizing from a single data point. Or at least, I do." - SA

When can you learn a lot from one data point? People, especially stats- or science- brained people, are often confused about this, and frequently give answers that (imo) are the opposite of useful. Eg they say that usually you can’t know much but if you know a lot about the meta-structure of your distribution (eg you’re interested in the mean of a distribution with low variance), sometimes a single data point can be a significant update.

This type of limited conc... (read more)

The significance, as I read it, is that you can now trust Claude roughly like a reasonable colleague for spotting such mistakes, both in your own drafts and in texts you rely on at work or in life.

I wouldn't go quite this far, at least from my comment. There's a saying in startups, "never outsource your core competency", and unfortunately reading blog posts and spotting conceptual errors of a certain form is a core competency of mine. Nonetheless I'd encourage other Forum users less good at spotting errors (which is most people) to try to do something like... (read more)

Recent generations of Claude seem better at understanding blog posts and making fairly subtle judgment calls than most smart humans. These days when I’d read an article that presumably sounds reasonable to most people but has what seems to me to be a glaring conceptual mistake, I can put it in Claude, ask it to identify the mistake, and more likely than not Claude would land on the same mistake as the one I identified.

I think before Opus 4 this was essentially impossible, Claude 3.xs can sometimes identify small errors but it’s a crapshoot on whether it ca... (read more)

The dynamics you discuss here follow pretty intuitively from the basic conflict/mistake paradigm.

I think it's very easy to believe that the natural extension of the conflicts/mistakes paradigm is that policy fights are composed of a linear combination of the two. Schelling's "rudimentary/obvious" idea, for example, that conflict is and cooperation is often structurally inseparable, is a more subtle and powerful reorientation than it first seems.

But this is a hard point to discuss (because it's in the structure of an "unknown known"), and I didn't interview... (read more)

I like Scott's Mistake Theory vs Conflict Theory framing, but I don't think this is a complete model of disagreements about policy, nor do I think the complete models of disagreement will look like more advanced versions of Mistake Theory + Conflict Theory. 

To recap, here's my short summaries of the two theories:

Mistake Theory: I disagree with you because one or both of us are wrong about what we want, or how to achieve what we want)

Conflict Theory: I disagree with you because ultimately I want different things from you. The Marxists, who Scott was or... (read more)

Good idea, I reposted the article itself here: https://forum.effectivealtruism.org/posts/GyenLpfzRKK3wBPyA/the-simple-case-for-ai-catastrophe-in-four-steps 

I've been trying to keep the "meta" and the main posts mostly separate so hopefully the discussions for the metas and the main posts aren't as close together.

I've now written it here, thanks for all the feedback! :) https://linch.substack.com/p/simplest-case-ai-catastrophe

The bets I've seen you post seem rather disadvantageous to the other side, and I believed so at the time. Which is fine/good business from your perspective given that you managed to find takers. But it means I'm more pessimistic on finding good deals by both of our lights.

Hmm right now this seems wrong to me, and also not worth going into in an introductory post. Do you have a sense that your view is commonplace? (eg from talking to many people not involved in AI)

Here's my current four-point argument for AI risk/danger from misaligned AIs. 

• We are on the path of creating intelligences capable of being better than humans at almost all economically and militarily relevant tasks.
• There are strong selection pressures and trends to make these intelligences into goal-seeking minds acting in the real world, rather than disembodied high-IQ pattern-matchers.
• Unlike traditional software, we have little ability to know or control what these goal-seeking minds will do, only directional input.
• Minds much better than humans at

What are people's favorite arguments/articles/essays trying to lay out the simplest possible case for AI risk/danger?

Every single argument for AI danger/risk/safety I’ve seen seems to overcomplicate things. Either they have too many extraneous details, or they appeal to overly complex analogies, or they seem to spend much of their time responding to insider debates.

I might want to try my hand at writing the simplest possible argument that is still rigorous and clear, without being trapped by common pitfalls. To do that, I want to quickly survey the field so I can learn from the best existing work as well as avoid the mistakes they make.

I often see people advocate others sacrifice their souls. People often justify lying, political violence, coverups of “your side’s” crimes and misdeeds, or professional misconduct of government officials and journalists, because their cause is sufficiently True and Just. I’m overall skeptical of this entire class of arguments.

This is not because I intrinsically value “clean hands” or seeming good over actual good outcomes. Nor is it because I have a sort of magical thinking common in movies, where things miraculously work out well if you just ignore tradeo... (read more)

Thanks! I agree the math isn't exactly right. The point about x^2 on the rationals is especially sharp.

The problem with calling it "the paradox of the heap" is to make it sound like an actual paradox, instead of a trivially easy connection re:tipping points. I wish I had a better terminology/phrase for the connection I want to make.

Thanks, the feeling is mutual. 

Happy holidays to you too.

I think your comment largely addresses a version of the post that doesn't exist. 

In brief:

I don't think I claimed novelty; the post is explicitly about existing concepts that seem obvious once you have them. I even used specific commonly known terms for them. 

Theory of mind, mentalization, cognitive empathy, and perspective taking are, of course, not actually "rare" but are what almost all people are doing almost all the time. The interesting question is what kinds of failures you think are common. The more opinionated y

I had the same initial reaction! I'd guess others would have the same misreading too, so it's worth rewriting. fyi @Yulia Chekhovska 

For Inkhaven, I wrote 30 posts in 30 days. Most of them are not particularly related to EA, though a few of them were. I recently wrote some reflections. @Vasco Grilo🔸 thought it might be a good idea to share on the EA Forum; I don't want to be too self-promotional so I'm splitting the difference and posting just a shortform link here:

https://linch.substack.com/p/30-posts-in-30-days 

The most EA-relevant posts are probably

https://inchpin.substack.com/p/skip-phase-3

https://inchpin.substack.com/p/aging-has-no-root-cause

https://inchpin.substack.com/p/leg... (read more)

There are a number of implicit concepts I have in my head that seem so obvious that I don't even bother verbalizing them. At least, until it's brought to my attention other people don't share these concepts.

It didn't feel like a big revelation at the time I learned the concept, just a formalization of something that's extremely obvious. And yet other people don't have those intuitions, so perhaps this is pretty non-obvious in reality.

Here’s a short, non-exhaustive list:

• Intermediate Value Theorem
• Net Present Value
• Differentiable functions are locally linear
• Th

Thanks, I find the polls to be much stronger evidence than the other things you've said.

My overall objection/argument is that you appear to selectively portray data points that show one side, and selectively dismiss data points that show the opposite view. This makes your bottom-line conclusion pretty suspicious. 

I also think the rationalist community overreached and their epistemics and speed in early COVID were worse compared to, say, internet people, government officials, and perhaps even the general public in Taiwan. But I don't think the case for them being slower than Western officials or the general public in either the US or Europe is credible, and your evidence here does not update me much.

Why does this not apply to your original point citing a single NYT article?

See eg traviswfisher's prediction on Jan 24:

https://x.com/metaculus/status/1248966351508692992 

Or this post on this very forum from Jan 26:

https://forum.effectivealtruism.org/posts/g2F5BBfhTNESR5PJJ/concerning-the-recent-2019-novel-coronavirus-outbreak 

I wrote this comment on Jan 27, indicating that it's not just a few people worried at the time. I think most "normal" people weren't tracking covid in January. 

I think the thing to realize/people easily forget is that everything was really confusing and there was just a ton of contentious deba... (read more)

I wrote a short intro to stealth (the radar evasion kind). I was irritated by how bad existing online introductions are, so I wrote my own!

I'm not going to pretend it has direct EA implications. But one thing that I've updated more towards in the last few years is how surprisingly limited and inefficient the information environment is. Like obvious concepts known to humanity for decades or centuries don't have clear explanations online, obvious and very important trends have very few people drawing attention to them, you can just write the best book review... (read more)

presupposes that EAs are wrong, or at least, merely luckily right

Right, to be clear I'm far from certain that the stereotypical "EA view" is right here. 

I guess really I was saying that "conditional on a sociological explanation being appropriate, I don't think it's as LW-driven as Yarrow thinks", although LW is undoubtedly important.

Sure that makes a lot of sense! I was mostly just using your comment to riff on a related concept. 

I think reality is often complicated and confusing, and it's hard to separate out contingency vs inevitable stories f... (read more)

eh, I think the main reason EAs believe AGI stuff is reasonably likely is because this opinion is correct, given the best available evidence^[1]. 

Having a genealogical explanation here is sort of answering the question on the wrong meta-level, like giving a historical explanation for "why do evolutionists believe in genes" or telling a touching story about somebody's pet pig for "why do EAs care more about farmed animal welfare than tree welfare." 

Or upon hearing "why does Google use ads instead of subscriptions?" answering with the history of the... (read more)

• Near-term AGI is highly unlikely, much less than a 0.05% chance in the next decade.

Is this something you're willing to bet on? 

Congrats Peter! This is very exciting! 

crossposted from https://inchpin.substack.com/p/legible-ai-safety-problems-that-dont

Epistemic status: Think there’s something real here but drafted quickly and imprecisely

I really appreciated reading Legible vs. Illegible AI Safety Problems by Wei Dai. I enjoyed it as an impressively sharp crystallization of an important idea:

1. Some AI safety problems are “legible” (obvious/understandable to leaders/policymakers) and some are “illegible” (obscure/hard to understand)
2. Legible problems are likely to block deployment because leaders won’t deploy until they’re sol

Some of the negative comments here gesture at the problem you're referring to, but less precisely than you had.

I wrote a quick draft on reasons you might want to skip pre-deployment Phase 3 drug trials (and instead do an experimental rollout with post-deployment trials, with option of recall) for vaccines for high diseases with high mortality burden, or for novel pandemics. https://inchpin.substack.com/p/skip-phase-3

It's written in a pretty rushed way, but I know this idea has been bouncing around for a while and I haven't seen a clearer writeup elsewhere, so I hope it can start a conversation!

Are the abundance ideas actually new to EA folks? They feel like rehashes of arguments we've had ~ a decade ago, often presented in less technical language and ignoring the major cruxes.

Not saying they're bad ideas, just not new.

This post had 169 views on the EA forum, 3K on substack, 17K on reddit, 31K on twitter.

Link appears to be broken.

This is great news; I'm so glad to hear that!!!

I wrote a field guide on writing styles. Not directly applicable to the EA Forum but I used some EA Forum-style writing (including/especially my own) as examples. 

https://linch.substack.com/p/on-writing-styles

I hope the article can increase the quality of online intellectual writing in general and EAF writing in particular!
 

x-posted from Substack

Now, of course, being vegan won’t kill you, right away or ever. But the same goes for eating a diet of purely McDonald’s or essentially just potatoes (like many peasants did). The human body is remarkably resilient and can survive on a wide variety of diets. However, we don’t thrive on all diets. 

Vegans often show up as healthier in studies than other groups, but correlation is not causation. For example, famously Adventists are vegetarians and live longer than the average population. However, vegetarian is importantly diffe

I have a lot of sympathy towards being frustrated at knee-jerk bias against AI usage. I was recently banned from r/philosophy on first offense because I linked a post that contained an AI-generated image and a (clearly-labelled) AI summary of someone else's argument^[1]. (I saw that the subreddit had rules against AI usage but I foolishly assumed that it only applied to posts in the subreddit itself). I think their choice to ban me was wrong, and deprived them of valuable philosophical arguments that I was able to make^[2] in other subreddits like r/Phi... (read more)

I compiled a list of my favorite jokes, which some forum users might enjoy. https://linch.substack.com/p/intellectual-jokes

Yeah I think these two claims are essentially the same argument, framed in different ways. 

I appreciate this article and find the core point compelling. However, I notice signs of heavy AI editing that somewhat diminish its impact for me.

Several supporting arguments come across as flimsy/obvious/grating/"fake" as a result. For example, the "Addressing the Predictable Objections" reads more like someone who hasn't actually considered the objections but just gave the simplest answers to surface-level questions, rather than someone who  deeply brainstormed or crowdsourced the objections to the framework. Additionally, the article's tendency to... (read more)

Thanks, appreciate the empirical note and graph on trendlines!

Preventing an AI takeover is a great way for countries to help their own people!

Tbh, my honest if somewhat flippant response is that these trials should update us somewhat against marginal improvements in the welfare state in rich countries, and more towards investments in global health, animal welfare, and reductions in existential risk. 

I'm sure this analysis will go over well to The Argument subscribers!

It's funny (and I guess unsurprising) that Will's Gemini instance and your Claude instance both reflected what I would have previously expected both of your ex ante views to be! 

lmao when I commented 3 years ago I said 

As is often the case with social science research, we should be skeptical of out-of-country and out-of-distribution generalizability. 

and then I just did an out-of-country and out-of-distribution generalization with no caveats! I could be really silly sometimes lol.

Re the popular post on UBI by Kelsey going around, and related studies:

I think it helped less than I “thought” it would if I was just modeling this with words. But the observed effects (or lack thereof) in the trials appears consistent with standard theoretical models of welfare economics. So I’m skeptical of people using this as an update against cash transfers, in favor of a welfare state, or anything substantial like that.

If you previously modeled utility as linear or logarithmic with income (or somewhere in between), these studies should be a update ag... (read more)