Ben_West

Head of Online (EA Forum, effectivealtruism.org, Virtual Programs) at CEA. Non-EA interests include chess and TikTok.

TikTok: @benthamite

We are probably hiring: https://www.centreforeffectivealtruism.org/careers

Sequences

EA Hiring
EA Retention

Topic Contributions

Comments

Hiring: How to do it better

Thanks for the thoughtful response!

My anecdotal experience with GMA tests is that hiring processes already use proxies for GMA (education, standardized test scores, work experience, etc.) so the marginal benefit of doing a bona fide GMA test is relatively low.

It would be cool to have a better sense of when these tests are useful though, and an easy way to implement them in those circumstances.

Money vs Talent: Putting numbers on the tradeoff?

Just wanted to nudge that I would find this write up very valuable. Even if the ranges are very wide, I often want to reference some sort of monetary estimate of the value of labor, and having a post like this to reference would be quite useful.

Hiring: How to do it better

Thanks for writing this! I've personally struggled to apply academic research to my hiring, and now roughly find myself in the position of who "Stubborn Reliance" is criticizing, i.e. I am aware of academic research but believe it doesn't apply to me (at least not in any useful way). I would be interested to hear more motivation/explanation about why hiring managers should take these results seriously. 

Two small examples of why I think the literature is hard to apply:

GMA Tests
If you take hiring literature seriously, my impression is that the thing you would most take away is that you should use GMA tests. It repeatedly shows up as the most predictive test, often by a substantial margin.[1]

But even you, in this article arguing that people should use academic research, suggest to not use GMA tests. I happen to think you are right that GMA tests aren't super useful, but the fact that the primary finding of a field is dismissed in a couple sentences seems worthy of note.

Structured versus unstructured interviews
The thing which primarily caused me to update against using academic hiring research is the question of structured versus unstructured interviews. Hunter and Schmidt 1998, which I understand to be the granddaddy of hiring predictors meta-analyses, found that structured interviews were substantially better than unstructured ones. But the 2016 update found that they were actually almost identical – as I understand it, this update is because of a change in statistical technique (the handling of range restriction).

Regardless of whether structured or unstructured interviews are actually better, the fact that the result you get from academic literature depends on a fairly esoteric statistics question highlights how difficult it is to extract meaning from this research.[2]

My impression is that this is true of psychology more broadly: you can't just read an abstract which says "structured > unstructured" and conclude something useful; you have to really dig into the statistical methods, data sources, and often times you even need to request materials which weren't made public to actually get a sense of what's going on.

I'm not trying to criticize organizational psychologists – reality just is very complicated. But I do think this means that the average hiring manager – or even the average hiring manager who is statistically literate – can't really get much useful information from these kinds of academic reviews.

  1. ^

    E.g. Hunter and Schmidt 1998: "The most wellknown conclusion from this research is that for hiring employees without previous experience in the job the most valid predictor of future performance and learning is general mental ability ([GMA], i.e., intelligence or general cognitive ability; Hunter & Hunter, 1984; Ree & Earles, 1992)"

  2. ^

    I briefly skimmed the papers you cited in favor of structured interviews. I couldn't immediately tell how they were handling range restriction; no doubt a better statistician than myself could figure this out easily, but I think it proves my point that it's quite hard for the average hiring manager to make sense of the academic literature.

You should join an EA organization with too many employees

On the sanity check: Reddit makes about four cents in revenue per user per month. It doesn't seem crazy to me that the average user gets two dollars of value per month, but a lot of this would depend on things like how many of their users are diehard versus casual users.

You should join an EA organization with too many employees

We are looking to hire, thanks! I put a link to our open positions at the bottom of the post.

You should join an EA organization with too many employees

My model was:

  • It's kind of unclear what a marginal unit is for a software company like Reddit, but let's just say it's one user of their software
  • Profit maximizing firms produce until marginal cost = marginal benefit
  • Marginal benefit is 50 times higher
  • Therefore firms will accept 50 times higher marginal cost
  • Let's suppose that the percentage of this additional marginal cost which goes to labor remains unchanged, resulting in 50 times more employees per user

30 seconds of thought can identify a bunch of problems with this model, but I think the underlying insight that firms would hire more labor is broadly correct. I would be interested to hear if people think this is wildly off though!

Some potential lessons from Carrick’s Congressional bid

Furthermore, I'm not sure the information value alone was worth the millions spent on this campaign by the EA community. The 'lessons learned' listed in this forum post seem obvious.

The post author doesn't say anything about having a special connection to the campaign. I assume the "value of information" argument is that campaign staff/insiders gained knowledge they couldn't have gotten otherwise, and I'm not sure this post would shed much light on that argument either way. 

As a relatively trivial example of learning not available from a Google search: the campaign presumably learned things like how many people would show up to make calls, how much money they could raise, etc.

Bad Omens in Current Community Building

Can you clarify why you think it's "borderline illegal"? I assume you are referring to GDPR, but I'm not aware of any reason why the normal "legitimate interest" legal basis wouldn't apply to group organizers.

EA Tours of Service

Hmm,  I think the question of whether employees should have objective requirements is somewhat orthogonal from the question of whether they should have a tour of service. For example, many salespeople have sales quotas, despite not being on a tour of service.

That being said: the alternative to having objective requirements is something like "you must fulfill the whims of your manager" and it's not obvious to me that this is actually better for job security.

Load More