Katja is widely respected amongst the rationalists and, according to Hive, she is one of the most followed/respected EA accounts. But she doesn't give off the same vibe as many impact olympians. She doesn’t have iron self-will, nor does she manage a huge team. She's hasn't got all the facts at her fingertips. But she has got something, I’m confident of that. How can I be more like her?
To understand her impact, let's consider the top things she's done:
- She ran surveys on AI researchers well before they were needed and has continued to run them
- She wrote an early blog on how we could slow down AI. This blog, I've heard, played a part in encouraging the Musk AI letter, which in turn inspired the “Existential Risks” AI letter.
- She thought about AI long before it was vogue, since about 2010
- She has a large track record of predictions
These actions seem impactful to me. And I guess someone should have paid $10mn in hindsight for the first 2, maybe more.
To me, Katja has a very low tolerance for incomplete stories. When she sees something that she doesn’t quite understand or that seems a bit off she struggles to pretend otherwise, so she says “how does that work?”. She doesn’t accept handwaving when discussing something, whether it be the simulation argument, how efficient flight is or the plot of Dune, part 2. She wants an unbroken chain of arguments she can repeat.
She also doesn’t mind admitting she doesn’t know the answer. In her living room she will turn to her friend Joe Carlsmith and ask “Wait, why are we worried about AI, again?” even though she’s been thinking about this for 15 years. Because at that moment it doesn’t fit for her and she has a high tolerance for embarrassed when it comes to truth. There is an deep resolve here - she doesn't get it, so she will ask until she does.
She works on the most important thing, slowly. If you are Elon Musk, maybe you can just work all the time. But I am not. And much as I love her, neither is Katja. She does not get an abnormal amount of work done per day. Instead, month in, month out, Katja works on what she thinks is most important. And eventually she gets the survey done, years ahead of when it’s needed.
There are lessons we can take from this. Just as we often talk about learning to code, or task management, I can become better at saying “wait that doesn’t work”. Here are some strategies that let me be more like Katja:
- Write it down - it’s harder to fool myself into thinking something makes sense if I have to read it rather than speak it
- “What is the best thing? How do I do that?” - this is hard to put into practice but an underrated prompt
- Give more examples - one feature of Katja’s writing is she loves to list things. I think more people should list every example in favour of their argument and every counterexample they can think of. Spend 5 minutes on each.
- Can I make a forecast of that? - I find fatebook.io useful for this. As I forecast more I learn how poor my judgement is. And I think it’s improving.
- Know when I am not capable - Katja is good at knowing when something is beyond her. When she hasn’t thought about something or when it’s a quantitative problem and she hasn’t worked on it carefully enough. She doesn’t always like the hard work but she knows when it needs to be done.
- If you have the right answer you can afford to be slow - in a world of often lurching acceleration, it’s easy to forget that if I just knew the right thing, then I could probably take years over it. More output is (usually) more better, but so is more accuracy.
- Have a distinction between what you currently understand and what you’d bet on - If Peter Wildeford and I disagree, he’s probably right, but that doesn’t mean I now understand. It is worth tracking the difference between what I actually think and what I would say if I had to bet on it. They might be very different.
For balance, there are costs to this mindset. Katja actively cultivates a feeling that if she makes errors there may be grave suffering, but that she must go on regardless. Sometimes she dreads meetings, or struggles with migraines. At times she has said she would prefer the most painful experiences of her life if her work would do itself that day. This breaks my heart.
I don’t see these costs as central but they are worth mentioning. If you are someone who struggles with overwork or neuroticism, maybe choose a different role model. Or ideally, take the good without falling into this trap.
Because for me, Katja is a wise guide on epistemics. Just as I can work more hours or use automation, I can take steps to ensure the work I do actually has good results. I can carefully look for the best thing and do it. And perhaps, I would recommend developing an inner Katja, to turn to you while you are mid-flow and, kindly, say “Sorry, I don’t understand that step.”
Article originally posted at https://nathanpmyoung.substack.com/
I had a mix of desires to up-vote and down-vote (I have done neither), which might possibly be helpful to express:
I think in general people find it easier to notice criticisms of things than appreciate positives. But I think having more 'picking out surprising positives' is a useful way of learning, and in addition leads to a much more appreciative environment than picking out negatives.
I worry that we currently have overly high standards for writing about positives because in addition to it being kind of tricky to notice them, there are other difficulties around things like dislike of glorifying people. My guess is that we could create a happier, more collaborative community if we had slightly lower standards for appreciation / noticing the positives type discussions.
The reason that I think some memetic immune response is appropriate to things of this shape is something like: if they became (semi-)normalized, it could become strategically correct for people trying to play social games to write puff pieces, and to get others to write puff pieces about them. I think it's better to live in a world where that isn't incentivized. So even though that isn't what I think is happening here, I don't think that will be reliably transparent to all readers, so I think it's maybe good for precedent-setting if the post gets some pushback on these grounds?
That said, I do feel good about more appreciating-positive type discussions (and think my above comment may sound too negative on anything about celebrating good things). I just want people to find ways that don't also create the bad incentives. Things I think might be helpful for this: primarily celebrating virtues, but also naming multiple people who do a job embodying the virtue; marrying discussion of positives with negatives (something which this post did some, which helps); creating canonical times for low-key sharing of positives (so that there's less information carried in the speech act of sharing, and more in the content of what's shared).
I agree it's sort of a red flag, but it seems relevant whether this is a puff piece, right?
Extremely relevant for my personal assessment!
For the social fabric stuff it seems more important whether it's legibly not a puff piece. Had I downvoted (and honestly I was closer to upvoting), the intended signal would have been something like "small ding for failing to adequately signal that it's not a puff piece" (such signalling is cheaper for things that actually aren't puff pieces, so asking for it is relatively cheap and does some work to maintain a boundary against actual puff pieces). It would have warranted a bigger ding if I'd thought it was a puff piece.
It's still possible I'm miscalibrated and this is transparently not a puff piece to ~everyone. (Although the voting pattern on my comment suggests that my feeling was not an outlier ... I guess this is unsurprising to me as one of the reasons I wrote the comment was seeing that your post had some downvotes and thinking it might be helpful to voice my guess about why.)
Yeah the tone makes sense for a personal blog (and in general the piece makes more sense for an audience who can mostly be expected to know Katja already).
I think it could have signalled more not-being-a-puff-piece by making the frame less centrally about Katja and more about the virtues you wanted to draw attention to. It's something like: those, rather than the person, are the proper object-of-consideration for these large internet audiences. Then you could also mention that the source of inspiration was the person.
Yeah that seems right. Not sure what options one can click on crossposting to point that out. (I think the forum has a personal blog option, but I'm not sure that's so appropriate on LessWrong)
Your post is tagged personal blog on LessWrong, idk if you tagged it that way explicitly or if it was done by mods.
For cross-posts to the EA forum, I think you might have an option in the ... menu at the top, or you can ask mods to move it to personal blog
Okay, this should be a personal blog then I think
Good point, I hadn't appreciated that. Thanks!
Those do seem like good compromises.
I guess it helps balance out the denouncement posts.
I quite liked it, but I'd happily give up praise posts if it meant not having the denouncement posts.
Though sometimes denouncement posts are net positive right? Like probably not the nonlinear one, but I guess more denouncement of SBF prior would have been good.