by Lizka1 min read9th Oct 202118 comments
18 comments, sorted by Click to highlight new comments since: Today at 12:06 PM
New Comment

Reflection on my time as a Visiting Fellow at Rethink Priorities this summer

I was a Visiting Fellow at Rethink Priorities this summer. They’re hiring right now, and I have lots of thoughts on my time there, so I figured that I’d share some. I had some misconceptions coming in, and I think I would have benefited from a post like this, so I’m guessing other people might, too. Unfortunately, I don’t have time to write anything in depth for now, so a shortform will have to do.

Fair warning: this shortform is quite personal and one-sided. In particular, when I tried to think of downsides to highlight to make this post fair, few came to mind, so the post is very upsides-heavy. (Linch’s recent post has a lot more on possible negatives about working at RP.) Another disclaimer: I changed in various ways during the summer, including in terms of my preferences and priorities. I think this is good, but there’s also a good chance of some bias (I’m happy with how working at RP went because working at RP transformed me into the kind of person who’s happy with that sort of work, etc.). (See additional disclaimer at the bottom.)

First, some vague background on me, in case it’s relevant:

  • I finished my BA this May with a double major in mathematics and comparative literature.
  • I had done some undergraduate math research, had taught in a variety of contexts, and had worked at Canada/USA Mathcamp, but did not have a lot of proper non-Academia work experience.
  • I was introduced to EA in 2019.

Working at RP was not what I had expected (it seems likely that my expectations were skewed).

One example of this was how my supervisor (Linch) held me accountable. Accountability existed in such a way that helped me focus on goals (“milestones”) rather than making me feel guilty about falling behind. (Perhaps I had read too much about bad workplaces and poor incentive structures, but I was quite surprised and extremely happy about this fact.) This was a really helpful transition for me from the university context, where I often had to complete large projects with less built-in support. For instance, I would have big papers due as midterms (or final exams that accounted for 40% of a course grade), and I would often procrastinate on these because they were big, hard to break down, and potentially unpleasant to work on. (I got really good at writing a 15-page draft overnight.)

In contrast, at Rethink, Linch would help me break down a project into steps (“do 3 hours of reading on X subject,” “reach out to X person,” “write a rough draft of brainstormed ideas in a long list and share it for feedback,” etc.), and we would set deadlines for those. Accomplishing each milestone felt really good, and kept me motivated to continue with the project. If I was behind the schedule, he would help me reprioritize and think through the bottlenecks, and I would move forward. (Unless I’m mistaken, managers at RP had taken a management course in order to make sure that these structures worked well — I don’t know how much that helped because I can’t guess at the counterfactual, but from my point of view, they did seem quite prepared to manage us.)

Another surprise: Rethink actively helped me meet many (really cool) people (both when they did things like give feedback, and through socials or 1-1’s). I went from ~10 university EA friends to ~25 people I knew I could go to for resources or help. I had not done much EA-related work before the internship (e.g. my first EA Forum post was due to RP), but I never felt judged or less respected for that. Everyone I interacted with seemed genuinely invested in helping me grow. They sent me relevant links, introduced me to cool new people, and celebrated my successes.

I also learned a lot and developed entirely new interests. My supervisor was Linch, so it might be unsurprising that I became quite interested in forecasting and related topics. Beyond this, however, I found the work really exciting, and explored a variety of topics. I read a bunch of economics papers and discovered that the field was actually really interesting (this might not be a surprise to others, but it was to me!). I also got to fine-tune my understanding of and opinions on a number of questions in EA and longtermism. I developed better work (or research) habits, gained some confidence, and began to understand myself better.

Here’s what I come up with when I try to think of negatives:

  • I struggled to some extent with the virtual setting (e.g. due to tech or internet issues). Protip: if you find yourself with a slow computer, fix that situation asap.
  • There might have been too much freedom for me— I probably spent too long choosing and narrowing my next project topics. Still, this wasn’t purely negative; I think I ended up learning a lot during the exploratory interludes (where I went on deep-dives into things like x-risks from great power conflict, but they did not help me produce outputs). As far as I know, this issue is less relevant for more senior positions, and a number of more concrete projects are more straightforwardly available now. (It also seems likely that I could have mitigated this by realizing it would be an issue right away.)
  • I would occasionally fall behind and become stressed about that. A few tasks became ugh fields. As the summer progressed, I think I got better about immediately telling Linch when I noticed myself feeling guilty or unhappy about a project, and this helped a lot.
  • Opportunity cost. I don’t know exactly what I would have done during the summer if not RP, but it’s always possible it would have been better.

Obviously, if I were restarting the summer, I would do some things differently. I might focus on producing outputs faster. I might be more active in trying to meet people. I would probably organize my daily routine differently. But some of the things I list here are precisely changes in my preferences or priorities that result from working at RP. :)

I don’t know if anyone will have questions, but feel free to ask questions if you do have any. But I should note that I might not be able to answer many, as I’m quite low on free time (I just started a new job).

Note: nobody pressured me to write this shortform, although Linch & some other people at RP did know I was doing it and were happy for it. For convenience, here’s a link to RP’s hiring page.

Thanks for writing this Lizka! I agree with many of the points in this [I was also a visiting fellow on the longtermist team this summer]. I'll throw my two cents in about my own reflections (I broadly share Lizka's experience, so here I just highlight the upsides/downsides things that especially resonated with me, or things unique to my own situation):

Vague background:

  • Finished BSc in PPE this June
  • No EA research experience and very little academic research experience
  • Introduced to EA in 2019

Upsides:

  • Work in areas that are intellectually stimulating and feel meaningful (e.g. Democracy, AI Governance).
  • Become a better researcher. In particular, understanding reasoning transparency, reaching out to experts, the neglected virtue of scholarship, giving and receiving feedback, and being generally more productive. Of course, there is a difference between 1. Understanding these skills, and 2. internalizing & applying them, but I think RP helped substantially with the first and set me on the path to doing the second.
  • Working with super cool people. Everyone was super friendly, and clearly supportive of our development as researchers. I also had not written an EA forum post before RP, but was super supported to break this barrier.

Downsides:

  • Working remotely was super challenging for me. I underestimated how significant a factor this would be to begin with, and so I would not dismiss this lightly. However, I think there are ways that one can remedy this if they are sufficiently proactive/agent-y (e.g. setting up in-person co-working, moving cities to be near staff, using Focusmate, etc). Also, +1 to getting a fast computer (and see Peter's comment on this).
  • Imposter syndrome. One downside of working with super cool, brilliant, hard working people was (for me) a feeling that I was way out of my depth, especially to begin with. This is of course different for everyone, but one thing I struggled to fully overcome. However, RP staff are very willing to help out where they can, should this become a problem.
  • Ugh fields. There were definitely times when I felt somewhat overwhelmed by work, with sometimes negative spirals. This wasn't helped by personal circumstances, but my manager (Michael) was super accommodating and understanding of this, which helped alleviate guilt.

If it's helpful, I might write-up a shortform on some of these points in more depth, especially the things I learnt about being a better researcher, if that's helpful for others.

Overall, I also really enjoyed my time at RP, and would highly recommend :)

(I did not speak to anyone at RP before writing this).

Thanks a lot for writing about your experiences, Lizka and Tom! Especially the details about why you were happy with your managers was really valuable info for me. 

Protip: if you find yourself with a slow computer, fix that situation asap.

Note to onlookers that we at Rethink Priorities will pay up to $2000 for people to upgrade their computers and that we view this as very important! And if you work with us for more than a year, you can keep your new computer forever.

I realize that this policy may not be a great fit for interns / fellows though, so perhaps I will think about how we can approach that.

I think we should maybe just send a new mid-end chromebook + high-end headsets  with builtin mic + other computing supplies to all interns as soon as they start (or maybe before), no questions asked. Maybe consider higher end equipment for interns who are working on more compute-intensive stuff and/or if they or their managers asked for it.

For some of the intern projects (most notably on the survey team?), more computing power is needed, but since so much of RP work involves Google docs + looking stuff up fast on the internet + Slack/Google Meet comms, the primary technological bottlenecks that we should try to solve is really fast browsing/typing/videocall latency and quality, which chromebooks and headsets should be sufficient for.

(For logistical reasons I'm assuming that the easiest thing to do is to let the interns keep the chromebook and relevant accessories)

I keep coming back to this map/cartogram. It's just so great. 

I tried to do something similar a while ago looking at under-5 mortality.

Superman gets to business [private submission to the Creative Writing Contest from a little while back]


“I don’t understand,” she repeated. “I mean, you’re Superman.”

“Well yes,” said Clark. “That’s exactly why I need your help! I can’t spend my time researching how to prioritize while I should be off answering someone’s call for help.”

“But why prioritize? Can’t you just take the calls as they come?”

Lois clicked “Send” on the email she’d been typing up and rejoined the conversation. “See, we realized that we’ve been too reactive. We were taking calls as they came in without appreciating the enormous potential we had here. It’s amazing that we get to help people who are being attacked, help people who need our help, but we could also make the world safer more proactively, and end up helping even more people, even better, and when we realized that, when that clicked—”

“We couldn’t just ignore it.”

Tina looked back at Clark. “Ok, so what you’re saying is that you want to save people— or help people — and you think there are better and worse ways you could approach that, but you’re not sure which are which, and you realized that instead of rushing off to fight the most immediate threat, you want to, what, do some research and find the best way you can help?”

“Yes, exactly, except, they’re not just better, we think they might be seriously better. Like, many times better. The difference between helping someone who’s being mugged, which by the way is awful, so helping them is already pretty great, but imagine if there’s a whole city somewhere that needs water or something, and there are people dying, and I could be helping them instead. It’s awful to ignore the mugging, but if I’m going there, I’m ignoring the city, and of those...”

“Basically, you’re right, Tina, yes,” said Lois.

“Ok,” Tina felt like she was missing something. “But Lois, you’re this powerful journalist, and Clark, you’re Superman. You can read at, what, 100 words per second? Doesn’t it make more sense for you to do the research? I’d need to spend hours reading about everything from food supply chains in Asia to, I dunno, environmental effects of diverting rivers or something, and you could have read all the available research on this in a week.”

“It’s true, Clark reads fast, and we were even trying to split the research up like that at some point,” said Lois. “But we also realized that the time that Clark was spending reading, even if it wasn’t very long, he could be spending chasing off the villain of the week or whatever. And I couldn’t get to all the research in time. I tried for a while, but I have a job, I need to eat, I need to unwind and watch Spanish soap operas sometimes. I was going insane. So we’ve been stuck in this trap of always addressing the most urgent thing, and we think we need help. Your help.”

“Plus, we don’t even really know what we need to find out. I don’t know which books I should be reading. It’s not even just about how to best fix the problem that’s coming up, like the best way to help that city without water. It’s also about finding new problems. We could be missing something huge.”

“You mean, you need to find the metaphorical cities without water?” Clark was nodding. Lois was tapping out another email. “And you should probably be widening your search, too. Not just looking at people specifically, or looking for cities without water, but also looking for systems to improve, ways to make people healthier. Animals, too, maybe. Aliens? Are there more of you? I’m getting off track.” Tina pulled out the tiny notebook her brother gave her and began jotting down some questions to investigate.

“So, are you in?” Lois seemed a bit impatient. Tina set the notebook aside, embarrassed for getting distracted.

“I think so. I mean, this is crazy, I need to think about it a bit. But it makes sense. And you need help. You definitely shouldn’t be working as a journalist, Clark. I mean, not that I’m an expert, really, but—”

“You kind of are. The expert.” Tina absently noted that Clark perfectly fit her mental image of a proper Kansas farm boy. He was even wearing plaid.

“If you accept the offer.” Lois said, without looking up from her email.

“That’s a terrifying thought. It feels like there should be more people helping, here. You should have someone sanity-checking things. Someone looking for flaws in my reasoning. You should maybe get a personal assistant, too— that could free up a massive amount of your time, and hopefully do a ton of good.” Tina knew she was hooked, but wanted to slow down, wanted to run this whole situation by a friend, or maybe her brother. “Can I tell someone about this? Like, is all of this secret?”

Clark shook his head. “We don’t want to isolate you from your friends or anything. But there will be things that need to be secret. And we’ve had trouble before— secrets are hard—” Clark glanced apologetically at Lois, who looked up from her frantic typing for long enough to shoot him a look, “But as much as possible, we don’t want to fall into bad patterns from the past.”

“I guess there are some dangers with information leaking. You probably have secret weaknesses, or maybe you know things that are dangerous—” Tina’s mind was swirling with new ideas and new worries. “Wait a second, how did you even find me? How do you know I’m not going to, like, tell everyone everything...”

Clark and Lois looked at each other.

“We didn’t really think that through very much. You seemed smart, and nice, and you’d started that phone-an-anonymous-friend service in college. And you wrote a good analysis when we asked you to. Sorry about the lie about the consulting job, by the way.”

“And you really need help.” Tina nodded. “Ok, we definitely need to fine-tune the hiring process. And I’ll start by writing down a list of some key questions.”

“I’ll order takeout,” said Lois, and pulled out her phone. 

 

[I wrote and submitted this shortly before the deadline, but was somewhat overwhelmed with other stuff and didn't post it on the Forum. I figured I'd go ahead and post it now. (Thanks to everyone who ran, participated in, or encouraged the contest by reading/commenting!]

I really liked this. It was simply, but a smooth read and quite enjoyable. I'd  be happy to see more of this type of content.

I recently ran a quick Fermi workshop, and have been asked for notes several times since. I've realized that it's not that hard for me to post them, and it might be relatively useful for someone.

Quick summary of the workshop

  1. What is a Fermi estimate?
  2. Walkthrough of the main steps for Fermi estimation
    1. Notice a question
    2. Break it down into simpler sub-questions to answer first
    3. Don’t stress about the details when estimating answers to the sub-questions
    4. Consider looking up some numbers
    5. Put everything together
    6. Sanity check
  3. Different models: an example
  4. Examples!
  5. Discussion & takeaways

Resources

Disclaimers:

  1. I am not a Fermi pro, nor do I have any special qualifications that would give me credibility :)
  2. This was a short workshop, aimed mostly at people who had done few or no Fermi estimates before

***************

***************

***************

***************

***************

***************

***************

***************

***************

***************

I attended and thoroughly enjoyed your workshop! Thanks for posting these notes

Thanks for coming to the workshop, and for writing this note!

I don’t see mention of quantifying the uncertainty in each component and aggregating this (usually via simulation). Is this not fundamental to Fermi? (Is it only a special version of Fermi, the “Monte Carlo” version?)

Uncertainty is super important, and it's really useful to flag. It's possible I should have brought it up more during the workshop, and I'll consider doing that if I ever run something similar.

However, I do think part of the point of a Fermi estimate is to be easy and quick.

In practice, the way I'll sometimes incorporate uncertainty into my Fermis is by running the numbers in three ways:

  1. my "best guess" for every component (2 hours of podcast episode, 100 episodes),
  2. the "worst (reasonable) case" for every component (only 90? episodes have been produced, and they're only 1.5 hours long, on average), and
  3. the "best case" for every component (150 episodes, average of 3 hours).

Then this still takes very little time and produces a reasonable range: ~135 to 450 hours of podcast (with a best guess of 200 hours). (Realistically, if I were taking enough care to run the numbers 3 times, I'd probably put more effort into the "best guess" numbers I produced.) I also sometimes do something similar with a spreadsheet/more careful Fermi.

I could do something more formal with confidence intervals and the like, and it's truly possible I should be doing that. But I really think there's a lot of value in just scratching something rough out on a sticky note during a conversation to e.g. see if a premise that's being entertained is worth the time, or to see if there are big obvious differences that are being missed because the natural components being considered are clunky and incompatible (before they're put together to produce the numbers we actually care about).

Note that tools like Causal and Guesstimate make including uncertainty pretty easy and transparent.

I really think there's a lot of value in just scratching something rough out on a sticky note during a conversation to e.g. see if a premise that's being entertained is worth the time

I agree, but making uncertainty explicit makes it even better. (And I think it's an important epistemic/numeracy thing to cultivate and encourage). So I think if you are giving a workshop you should make this part of it at least to some extent.

I could do something more formal with confidence intervals and the like

I think this would be worth digging into. It can make a big difference and it’s a mode we should be moving towards IMO, and should this be at the core of our teaching and learning materials. And there are ways of doing this that are not so challenging.

(Of course maybe in this particular podcast example it is now so important but in general I think it’s VERY important.)

“Worst case all parameters” is very unlikely. So is “best case everything”.

See the book “how to measure everything” for a discussion. Also the Causal and Guesstimate apps.

Here are slides from my "Writing on the Forum" workshop at EAGxBerlin.