All of geoffrey's Comments + Replies

Until recently, I always had the impression that there was a glut of animal activists and there'd be little point in me participating. It's not something I ever bothered to check.

For one thing, I heard plenty of stories of how hard it is to get a job at an animal organization. So I figured that would be same for animal activism and that each campaign was saturated with volunteers.

And for another thing, I usually don't hear about pressure campaigns unless they're successful or have tons of people. Understandably nobody wants to promote the mediocre attempts... (read more)

Don’t have answers but just wanted to say I really appreciate this mini-compilation of what’s already been done 

For what it's worth, I read that abstract as saying something like, "within the class of interventions studied so far, the literature has yet to settle onto any intervention that can reliably reduce animal product consumption by a meaningful amount, where meaningful amount might be a 1% reduction at Costco scale or long-term 10% reduction at a single cafeteria. The class of interventions being studied tends to be informational and nudge-style interventions like advertising, menu design, and media pamphlets. When effect sizes differ for a given type of inte... (read more)

Seth, for what it's worth, I found your hourly estimates (provided in these forum comments but not something I saw in the evaluator response) on how long the extensions would take to be illuminating. Very rough numbers like this meta-analysis taking 1000 hours for you or a robustness check taking dozens / hundreds of hours more to do properly helps contextualize how reasonable the critiques are.

It's easy for me (even now while pursuing research, but especially before when I was merely consuming it) to think these changes would take a few days. 

It's al... (read more)

Would you recommend that I share any such posts with both the authors and the evaluators before making them?

Yes. But zooming back out, I don't know if these EA Forum posts are necessary.

A practice I saw i4replication (or some other replication lab) is that the editors didn't provide any "value-added" commentary on any given paper. At least, I didn't see these in any tweets they did. They link to the evaluation reports + a response from the author and then leave it at that.

Once in a while, there will be a retrospective on how the replications are going as a... (read more)

Chiming in here with my outsider impressions on how fair the process seems

@david_reinstein If I were to rank the evaluator reports, evaluation summary, and the EA Forum post in which ones seemed the most fair, I would have ranked the Forum post last. It wasn't until I clicked through to the evaluation reports that I felt the process wasn't so cutting.

Let me focus on one very specific framing in the Forum post, since it feels representative. One heading includes the phrase "this meta-analysis is not rigorous enough". This has a few connotations that you pro... (read more)

Really enjoyed this. Not much public debate in this space as far as I can see. To two of your cruxes:

Is meta-analysis even useful in these contexts, with heterogeneous interventions, outcomes, and analytical approaches?

Will anyone actually do/fund/reward rigorous continued work?

I've sometimes wondered if it'd be worth funding a "mega study" like Milkman et al. (2021). They tested 54 different interventions to boost exercise among 61,000 members. Something similar for meat reduction could allow for some clean apples-to-apples comparisons.

I've seen the numbe... (read more)

Ooh this is neat.

I like how it neutralizes the certainty-seeking part of me since it's only me, the difference maker, that has the option of a guaranteed 100% outcome. For the beneficiary, it's always a gamble.

Agree it's more about upbringing and messaging. And also relate a lot to this.

But also I think it's really hard to tell the "cause" of any given problem at an individual level. As recently as a few years ago, I would have put 80% weight on upbringing / messaging (which I agree aren't the identities themselves but something associated with them). Nowadays I'm more agnostic about it.

I think it's fine to seek out affinity groups and culturally-relevant advice to some degree. But also, there's a tradeoff between exploring identities versus applying generic mental health advice. Especially when you get to intersectionality-type stuff like trifectas where the number of things to explore is gets incredibly vast very quickly.

I can speak to two of those three identities (EA and Asian). I think one possibility that took me an unusually long time to consider was that maybe my identities didn't matter and I'd still feel the same problems if I was the "default person" in society. And I was working through a lot of identities.

It's a weird way of framing things since we can't have our identities counterfactually removed. Even if we did, we wouldn't be the same person. But I think it's a framework that usually doesn't get mentioned much in mental health circles , especially on the int... (read more)

Agree the value is high. But practically, there's two big questions that pop to mind since I work / study around this area:

• If aggregating existing datasets, what's your value-add over what J-PAL, World Bank, IDInsight, Our World in Data, and what numerous un-affliated academics are already doing? (See Best of EconTwitter Substack for "public goods" which are sometimes publicly accessible datasets)
• If gaining access to new datasets, what are you offering to LMIC governments in exchange? Even making a single batch of batch of data publicly accessible is a lif

This is really good. 

What struck me was all the concrete detail. While it is personal, it's also in service of giving useful lessons to other people. It helps establish how generalizable the career advice would be to other people and it reframes some standard career advice in a way that centers the constraints as a first-order consideration.

I would not have taken the adversity quotient framing seriously otherwise.

The one addition that might help is mentioning whether there were aspects of your career path that felt unusually lucky or aspects of your l... (read more)

Do any of you have heuristics for when to “give up” or “pivot” in a job search? Examples could be aiming lower / differently if no response after 10 applications.

Thankfully this is not something I have to worry about for a long time. But I think it’s useful to have some balance to the usual advice of “just keep trying; job searching takes a long time”. Sometimes a job really is unrealistic for a person’s current profile (let’s operationalize that as 1000 job searching hours would still only result in a 1% chance of getting a certain set of jobs).

Thanks for this. I'm surprised how consistently the studies point in favor of vegan diets being cheaper on the whole (though I'll caveat none of these are too convincing: the headline RCT is testing a low-fat vegan diet instead of a general vegan diet and the rest are descriptive regressions / modeling exercises).

All that said, I'm wondering if perception of vegan diets being more expensive could be explained by:

• Fully plant-based diets are cheaper but various "halfway points" are more expensive.
• Meat-eaters mostly get exposure to the "halfway points". These

Agreed, but I'd be careful not to confuse good mentorship with good management. These usually go hand-in-hand. But sometimes a manager is good because they sacrifice some of your career growth for the sake of the company.

I like the archetypes of ruthless versus empathetic managers described here. It's an arbitrary division and many managers do fulfill both archetypes. But I think it also captures an important dynamic, where managers have to tradeoff between their own career, your career, and the organization as a whole. Mentorship and career development fa... (read more)

Not sure what the right numbers are but I really like the back-of-the-envelope approach you're taking here. It's simple and concrete enough that it's probably going to bounce around in my head for a while

Good point. In a toy model, it'd depend on relative cuts to labor versus non-labor inputs. Now that I think about it, it probably points towards exiting being better in mission-driven fields. People are more attached to their careers so the non-labor resources get cut deeply while all the staff try to hold onto their jobs.

Maybe I'd amend it to... if you're willing to switch jobs, then you can benefit from increasing marginal returns in some sub-cause areas. Because maybe there's a sub-cause area where lots of staff are quitting (out of fear the cause area ... (read more)

Marginal returns to work (probably) go up with funding cuts, not down.

It can be demoralizing when a field you’re working in gets funding cuts. Job security goes down, less stuff is happening in your area, and people may pay you less attention since they believe others are doing more important work. But assuming you have job security and mostly make career decisions on inside views (meaning you’re not updating too heavily on funders de-prioritizing your cause area), then your skills are more valuable than they were previously.

Lots of caveats apply of course... (read more)

My immediate hesitation is whether fresh college graduates would be useful enough to hosting organizations to make this program sustainable.

Last I checked, Peace Corps invests 3 months of formal training into each applicant and requires a minimum 2-year commitment in a role (to allow people to grow into competency). But this version of Animal Advocacy Corps has college undergraduates rotate thru multiple organizations for much shorter periods without any training. And I’m not sure how much demand there is for that kind of worker in animal advocacy even if it’s provided for free.

Agreed. I’d extend the claim past ideas and say that EA is very non-elitist (or at least better than most professional fields) at any point of evaluation. 

Maybe because of that, it seems more elitist elsewhere. 

Like the idea but it might be at odds with the recent AI governance shift. In general, policy folks of all stripes, especially the more senior and insider ones, practice a lot of selective disclosure.

Having done a lot of this advice in my 20s, I'd recommend just getting started with an online training program you find interesting, seems career relevant, and also not too pie-in-the-sky as a near-term plan. Throughout my life, I think there were one or two that felt unusually good or bad all-things-considered. Even then, training programs are short (~6 weeks) and have no stakes if you stop them.

(The exception is if the training somehow includes hands-on training from someone actively trying to progress in one of your desired career paths. Good mentorship... (read more)

I've done a lot of partially blind hiring processes both within EA and outside it ^[1]. And as much as I like them (and feel like I've benefited from them), I think there's good reasons why they aren't done more.

• It seems really hard to create a good blind hiring process. Most of the ones I felt good about were constructed with an immense amount of care to balance not rejecting good candidates but still having enough subtlety to distinguish candidates that would pass a final stage. Even then I still felt like there would be exceptional candidates that w

A quick drive-by comment on "4. Missed RCT Opportunity": The sample size seems way too small for a RCT to be worth it. There's not much statistical power to work with when researchers are studying a messy intervention with only 6 countries. And I imagine they'd struggle to attribute changes to the Technical Support Units unless it was something truly transformative (at least within the framework of the RCT).^[1]

More broadly, I'm not aware of any commonly accepted way to do "small n" impact evaluation yet, especially with something as customized as Technical... (read more)

Yeah this was what I found too when I looked into private US long-term disability insurance a while back. My recollection was:

•  there's a surprising number of health exclusions, even for things that happened in your childhood or adolescence
• it's a lot more expensive in percentage terms if you're at a lower income
• many disability cases are ambiguous so the insurance company may have you jump through a lot of hoops and paperwork (a strange role-reversal in which the bureaucracy wants to affirm your agency)

I had the impression that it was a great product fo... (read more)

I find searching for in-depth content on the EA Forum vastly better than Reddit. This isn't just relating to EA topics. There are a few academic-ish subreddits that I like and will search when I'm interested in what the amateur experts think on a given topic. Finding relevant posts is about the same on Reddit but finding in-depth comments + related posts is very hard. I usually have to do some Google magic to make that happen.

Also on rare occasion, I end up liking a person's writing style or thinking methods and want to deep dive into what else they've wri... (read more)

I'm loving this series so far. I got two questions if you've got some time to answer them.

What categories do you use for time-tracking? I find research tasks unusually hard to categorize.

Do you find that earlier stages in the Ideation -> Exploration -> Understanding -> Distillation pipeline take more time to get good at? My experience is that I improve at later stages far earlier and far faster than earlier stages (passable at Distillation before Understanding, passable at Understanding before Exploration, passable at Exploration before Ideation). And anecdotally, I heard people can take a very long time to come up with a good research idea.

I found it very valuable but (barring any major changes like there being 5 new organizations of similar impact) I wouldn’t find an expanded or updated version that useful.

Interesting! I think my intuition going into this has always been stretching so that's something I could rethink

Ah I missed the point about the relationship getting flatter before. Thanks for flagging that.

I think I'm more confused about our disagreement now. Let me give you a toy example to show you how I'm thinking about this. So there's three variables here:

• latent life satisfaction, which ranges from 0 to infinity
• reported life satisfaction, which ranges from 0 to 10 and increases with latent life satisfaction
• probability of divorce, which ranges from 0% to 100% and decreases with latent life satisfaction

And we assume for the sake of contradiction that rescaling is... (read more)

This is really neat analysis idea. 

At the same time, my hunch is that all three of these exit actions have gotten easier to do and more common from 1990 to 2022. I believe divorce has gotten less stigmatized, the job market rewards more hopping around, and (I think) hospitalization has been recommended more.

If that "easier-to-do" effect is large enough, then it'd be compatible with a very wide range of happiness trends (rising/falling/stable + rescaling/no rescaling) Wondering if you have any thoughts on that.

Hoping to post some shallow takes on global health and development this year!

This is incredible. I skimmed all the sections and I'm impressed with the quality, scope, concreteness, and kindness throughout. 

This is an area where Probably Good blows the 80,000 Hours version out of the water. I'll be pointing people here for a one-stop shop for all their job searching needs and almost certainly coming back to this in the future.

Quick anecdote: I found this dynamic surprising in all the paths you mentioned: academia, research, EA research, and non-profit work. But I realized it very quickly for academia (one month) and painfully slowly for non-profits (several years).

In academia, the markers of success are transparent, the divide between "good" and "bad" jobs is sharp, and even very successful professors complain about the system.

On the other extreme, the non-profit sector is much fuzzier about credentials and career progression. So I might see an employee who graduated from a sch... (read more)

Hey Ozzie, a few quick notes on why I react but try not to comment on community based stuff these days:

• I try to limit how many meta-level comments I make. In general I’d like to see more object-level discussion of things and so I’m trying (to mixed success) to comment mostly about cause areas directly.
• Partly it’s a vote for the person I’d like to be. If I talk about community stuff, part of my headspace will be thinking about it for the next few days. (I fully realize the irony of making this comment.)
• It’s emotionally tricky since I feel responsibility for

I also share these frustrations with career advice from 80,000 Hours and the EA Forum. There was time about 2 years back where my forum activity was a lot of snarky complaints (of questionable insight) about career advice and diversity.

Like you mentioned, the career advice usually leaves a lot to be desired in the concrete details of navigating a lack of mentors, lack of credentials, lack of financial runway, family obligations, etc. I've sometimes wondered about writing an article to fill in the gap, but it's not exactly a "one article" sized hole. Maybe ... (read more)

This is great stuff. I often find it hard to remember a lot of initiatives have happened (despite having read 80% of this list already) so this timeline is a good reference

As an aside, I think others may benefit from reading about diversity initiatives outside EA to remember this is hard problem. It's totally consistent for EA to be above-the-curve on this and still not move the needle much (directionally I think those two things are true but not confident on magnitudes), so linking some stuff I've been reading lately:

• DEI Deconstructed: Your No-Nonsense Gu

Echoing what Eva said, I think you should consider waiting a year then apply for IDE / applied econ masters. An IDE program is probably the right fit given your goals, but I don't know any beyond Yale's IDE which expects you to already have worked in development first.

For Applied Econ, I like University of Maryland's Applied Economics Master's program. The program only requires Calc I and is very transparent about what it can do. Dev / global health placements, content, and networking will take a huge hit compared to IDE programs though.

You can use the yea... (read more)

This is my favorite in the series so far. I really enjoy the tacit knowledge flavor of it and that some of these lessons generalize beyond the CBA domain.

Would also love this. I think a useful contrast will be A/B testing in big tech firms. My amateur understanding is big tech firms can and should run hundreds of “RCTs” because:

• No need to acquire subjects.
• Minimal disruption to business since you only need to siphon off a minuscule portion of your user base.
• Tech experiments can finish in days while field experiments need at least a few weeks and sometimes years.
• If we assume treatments are heavy tailed, then a big tech firm running hundreds of A/B tests is more likely to learn of a weird trick that grows the business when compared to a NGO who may only get one shot. 

Something I've noticed more in the EA Forum is the increase in drive-by professional posts. Organizations will promote a idea, a job posting, or something else. Then they'll engage as long as they're on the front page before bouncing.

That's fine in small amounts or if the author is a regular contributor. But if the author is just stopping by to do their public engagement, then it breaks the illusion of a community.

And for me, that is the aesthetic draw of the forum. It's a place where expects and amateurs alike coexist in the same space, say things that ar... (read more)

This is a class act in reasoning transparency. I love how easy it is to skim and drill down into things for more detail. Same goes for the pre-print and replication code.

Nits:

• I was confused what calibration meant since I think of this exercise as simulation. To me, calibration is taking some observed data and reverse engineering the elasticity. But this is starting with the elasticity values -- taken from the Bouyssou et al. (2024) meta-analysis -- and seeing how that goes forward in a pretend-tax situation.
• I would have loved to hear about how trustworthy

I love this tiny hearts on pink banner aesthetic. Especially in the middle of winter where I'm at. 

I'm on a pause with donations for some personal and financial reasons, but if I was on the fence or was procrastinating, this absolutely would have pushed me to do so.

Personal reasons why I wished I delayed donations: I started donating 10% of my income about 6 years back when I was making Software Engineer money. Then I delayed my donations when I moved into a direct work path, intending to make up the difference later in life. I don't have any regrets about 'donating right away' back then. But if I could do it all over again with the benefit of hindsight, I would have delayed most of my earlier donations too.

First, I've been surprised by 'necessary expenses'. Most of my health care needs have been in therapy and denta... (read more)

Hey John, this is very cool to read. I like the focus on what surprised you as a founder (and maybe newcomer?) in the mental health field.

I'm curious to hear more about the implementation details. Could you tell me more about the length, intensity, and duration of a typical treatment program? I saw 6 sessions in a graph which makes me think this is once-a-week program for 1-2 hour sessions over 1-2 months

Less sessions is a reliable way to reduce cost, but my understanding is there’s a U-shaped curve to cost-effectiveness here. 1 session doesn't have enough... (read more)

Quickly throwing in a related dynamic. I suspect animal welfare folks have more free time to post online.

Career advancement in animal welfare is much more generalist than global health & development. This means there's not as many career goals to 'grind' towards, leaving more free time for public engagement. Alternative proteins feel like a space where one can specialize, but that's all I can think of. I'd love to know of other examples.

In contrast, global health & development has many distinct specialities that you have to focus on if you want to ... (read more)

Thanks, this is exactly what I'm looking for. 

Accuracy isn't too important here. More interested in how people approach this

This advice sounds right to me if you already have the signal in hand and deciding whether to job search.

But if you're don't have the signal, then you need to spend time getting it. And then I think the advice hinges on how long it takes to get the signal. Short time-capped projects are great (like OP's support on 80,000 hours CEO search). But for learning and then demonstrating new skills on your own, it's not always clear how long you'll need.

Ooh good idea. I should do more of that.

I do think this can run into Goodhart's Law problems. For example, in the early 2010s, back when being a self-taught software engineer was much more doable, and it was a strong sign when someone had a GitHub profile with some side projects with a few months of work behind each of them. GitHub profile correlated with a lot of other desirable things. But then everyone got that advice (including me) and that signal got rapidly discounted.

So I guess I'd qualify that with: press really hard on why the signal is impressive... (read more)

I like this advice a lot but want to add a quick asterisk when transitioning to a new field.

It’s really really hard to tell what an expensive signal is without feedback. If you’re experienced in a field or you hang out with folks who work in a field, then you’ve probably internalized what counts as an “impressive project” to some degree.

In policy land, this cashes out as advice to take a job you don’t want in the organization you do want. Because that’s how you’ll learn what’s valuable and what’s not. Or taking low paid internships and skilled volunteering... (read more)