Comment Permalink

On (1), the organization needs to clearly explain that this is a task that is used for assessment and that the organization will not benefit from the task. Otherwise, the candidate will likely walk away feeling used, and may share information about the trial task with others (why wouldn't they if they were not aware it was a standardized instrument that will be reused?)

I have mixed feelings about being "selective with the candidates that get asked to take a trial" -- if you weight performance on the trial very highly, then trimming the number of candidates able to take a trial denies people who may have scored very well (but whose resume is middle-of-the-road) a chance to prove themselves. Obviously, one should be more selective with invitations to more time-consuming and arduous trials.

See in context

Opinion: EA-jobs interviews are used by organizations/companies as free labor

by Vaipan

Dec 21 20222 min read 23

5

CommunityHiringWorking at EA vs. non-EA orgs

Frontpage

So this is an empirical post to warn others and to discuss these practices. I think it's essential, especially since many orgzs/companies have people writing and reading this forum. I am not writing out of bitterness; just stating what I've experienced. I hope they will engage in this conversation!

A bit of context: I am finishing a Ph.D. in History. An underdeveloped discipline in the EA field, although McAskill's recent book might have a good effect on changing this. So, I'm trying to sell my research skills in a more reality-anchored, useful field than medieval state-building (happy to chat about it if that sounds interesting!).

I have applied to many, many jobs. The success rate being 5% (a number I heard during a 'celebrate our failure' gig), I am not surprised to struggle. However, the most frustrating thing is being chosen for the first, second, even third, or fourth or fifth (last one I had) interview round, completing tasks for free more than half of the time, and just....raising my hopes for nothing as I'm never chosen in the end.

Let me explain. I had to fulfill very practical and useful tasks for the organizations to which I applied. Without naming institutions, I'll cite one where I had to choose relevant candidates to speak for a panel or another where I had to devise a whole plan to manage a new research unit. I spent hours on these tests. My most rational approach was to do my best because I knew competition to be fierce and that I couldn't just mildly apply for something and have a single chance to be reinvited for an interview without giving my best.

As I'm finishing my thesis, it took much time and energy of thought to complete these tests. It took hope. And yet, I was barely compensated for my time despite giving back, what I believe, was work of very high quality. I've never seen this phenomenon outside of EA, e.g., the requirement of working on intellectually intensive tasks that take more than a few hours. Maybe it exists: but it seems to me that because EA-related companies are more likely to raise feelings of hope and admiration than those that are not EA-oriented (in sum, those that only exist for profit without any other aim), EA-related companies invite more people than what they anticipate to employ to complete some labor for free.

And it doesn't feel right. I know EA isn't (directly) about unions and fair rights for employees, but it feels that organizations use my work without them having the intention to hire me.

Thoughts?

5 Reactions

Comments23

Sorted by

New & upvoted

Click to highlight new comments since: Today at 4:39 PM

Peter Wildeford2y43

I'm really sorry you had such negative experiences with your applications.

I'd like to state for the record that Rethink Priorities makes heavy use of work tests because we find they best correlate with on-the-job performance.

However, we:

(1) always ensure generous and competitive compensation for work tests done regardless of whether the applicant moves on in the process

and (2) never make actual use of work test output other than to assess the applicant (e.g., we only ask questions that we've already researched internally)

While I'm sympathetic to organizations who may not have the funding, I think it would make sense in the future for candidates to refuse to engage with multi-hour work trials without fair compensation.

Jonas V2y10

Same for the organization I run.

Kaleem2y22

Hi Vaipan, thanks for writing this. As someone who's applied for (many) dozens of EA and non-EA jobs over the past 2 years, I can identify and empathise with your story here.

From my experience, I think EA orgs do use 'work tests' way more often than non-EA orgs (I very rarely had to do a work test for non-EA job hiring, only interviews). There are obviously pros and cons to using work tests, and in my short time at a couple of different EA orgs, I've been part of pretty frequent discussion about how/when/why we should use work tests in hiring rounds (to their credit, I think).

The main thing that I'm surprised by in your post is the high frequency that you report to have done unpaid work tests. In recent times I've been really happy to see most EA orgs offering compensation for work tests, and I think the vast majority of work tests that I did were compensated.

"but it feels that organizations use my work without them having the intention to hire me. "

^This quote is pretty concerning .In general, I don't think I've seen anyone use the product of work tests for anything other than assessing them for hiring purposes. Are you saying that orgs have used the product of your work tests for other purposes? In that case, it does seem especially bad for that work test to have been uncompensated, and I'd assume (and hope) that most other EA's would agree with me.

Vaidehi Agarwalla 🔸2y15

If you don't feel comfortable sharing which orgs they were publicly, I'd strongly recommend speaking to the community health team at CEA about this.

Vaipan2y2

Hi Kaleem!

I hope you have found your fit by now! No, I don't have any evidence that my work has been used, but the assignments were so specific that it gave me this impression. However, it is only an impression!

Luke Freeman 🔸2y15

Some anecdata:

I’ve done (or asked to do and then subsequently refused) several extensive work tests in job applications prior to working with the EA ecosystem. None of these were paid.
I’ve almost always had candidates do paid work tests for the final round (before interview and reference checks) of roles I’ve hired for within my role as ED of GWWC and on the board at EA Australia.
I’ve not used the work generated from work tests as free labour (on a couple of instances I asked candidates who did exceptionally in the work test but didn’t get the role if they wanted their work to be used and paid them to complete it as a contractor and in both those cases I was a reference for them for other roles they ended up getting).

I’ve found work tests to be very worthwhile in hiring (often the single most predictive) . In return (as well as paying) I’ve offered feedback to all candidates who’ve done the work test.

I’m glad you raised this and while I’m not sure of the prevalence of the problem within the EA ecosystem I think it’s important we have norms around paying people for their work (unless it is explicitly charitable volunteering which is something that I think is good to provide opportunities for people to do and has been valuable in my own impact journey).

Vaipan2y3

Hi Luke,

I'm happy to know that your org has better practices. It's also the feedback part that sucks: despite succeeding at many rounds, I never gotten any feedback!

Joel Becker2y13

The success rate being 5% (a number I heard during a 'celebrate our failure' gig), I am not surprised to struggle.

FWIW, I found this surprisingly high, assuming that the denominator is something like: "all candidates applying to all jobs on 80k board," which excludes internal hiring/closed hiring/etc.

technicalities2y10

I personally only offer paid work trials, and this is the norm in the orgs I've seen (e.g. OpenPhil). I hope the answer is that the ones you experienced actually can't afford to do this (but I'm sure some could).

Oscar Howie2y9

I’m sorry to hear you’ve had these negative experiences.

At CEA, trial tasks (minutes or hours) and work trials (days) are key components of our typical hiring rounds because we believe them to be better predictors of performance in the role than other assessments such as resumes or interviews. We do, however, always offer to compensate candidates for their time for any longer trial tasks and work trials if it's legally possible. (Visa rules mean this is sometimes not possible for candidates who travel, e.g. to the UK, for a trial without a work visa.)

kokotajlod2y9

I think work tests are a great way to hire people (because they are less biased than the alternatives) but I agree they should be paid for. I didn't know unpaid work tests were a thing. Which orgs had extensive unpaid work tests?

Kirsten2y8

I think work tests depend on the industry. For example when I was working as a teacher, my interview usually included teaching a real class of students, but in the Civil Service I've never been asked to do anything more than prepare a presentation.

Will Bradshaw2y7

Following other commenters here, I'm heavily pro work tests, but always advocate strongly for (what I believe to be) fair compensation. If organisations are having you do major unpaid (or e.g. minimum-wage) work trials, that seems like a serious problem.

There are also two broad categories of paid work tests I've seen, and I think the distinction between them is important:

The first is standardised "trial tasks", in which every candidate is given an identical task that functions like an exam. These are typically purely for assessment and have little to no practical benefit to the organisation; they also typically come earlier in the process.
The second is an on-the-ground "work trial" in which the candidate works closely with members of the hiring org over one to a few days. These trials more commonly involve work that's useful to the organisation, since the goal is to see how the candidate works in a more realistic setting with their potential new teammates. These typically come later in the process.

In my opinion, both of these should be paid, with the importance of providing payment increasing with the length and arduousness of the trial. I also think (2) should only be used with a small number of finalist candidates the org is genuinely excited about. If an org is instead giving these kinds of trials to significant numbers of candidates with lower probability of hiring, then I agree that seems quite problematic, especially if the trials aren't fairly compensated (the need for fair compensation also creates an incentive to be selective with the candidates that get asked to take a trial, which I think is helpful for reducing the concerns raised here).

Jason2y2

Will Bradshaw2y6

Otherwise, the candidate will likely walk away feeling used, and may share information about the trial task with others

We generally try to be pretty clear that this is a standardised test, yeah. Including asking people not to share it with others.

I have mixed feelings about being "selective with the candidates that get asked to take a trial" -- if you weight performance on the trial very highly, then trimming the number of candidates able to take a trial denies people who may have scored very well (but whose resume is middle-of-the-road) a chance to prove themselves.

I agree that this is a tradeoff -- one that is made sharper by the expense of paying fairly for trial tasks (if we didn't pay, we could have a lower bar for sending out trial tasks). I think it's fairly common to try to address this by having multiple trial tasks / work trials of increasing length & selectivity. (Though this has its own cost, namely an unusually lengthy and gruelling application process.)

Jason2y2

Do all candidates necessarily start at the same stage of trial tasks in most organizations? I could imagine a very short "stage 0" trial task requested only from candidates who would be on the bubble of moving on to the first, moderately time-consuming task if there were no "stage 0" task to evaluate. The "stage 0" task would only need enough power to tip someone on the bubble over into either the "it's respectful to ask them to do a more demanding task" or "respect their time and move on" buckets. Of course, you'd need to find a quick "stage 0" task that correlates well enough with either job performance or later-stage test performance.

Yonatan Cale2y4

Ideally there'd be a supervisor-org where you could tell them about this, nothing bad would happen to you, and they'd check with the hiring-org what's going on.

With all the discussion on EA governance lately, this is something that seems relevant to me

Jeff Kaufman 🔸2y16

This could be something to bring to the CEA community health team? It's not a central example of the kind of case I think they normally handle, but it does seem like a kind of "EAs treating each other badly".

(Disclosure: I'm married to Julia Wise, but that doesn't mean I know whether the team would consider this within their scope)

Yonatan Cale2y2

Thanks! I'll refer this comment to relevant people

Guy Raveh2y2

Why an org and not a public platform?

Yonatan Cale2y9

I'm not against

I don't really know how to set up a structure like this, I assume some people thought a lot about such topics

lastmistborn2y3

I've come across the same thing in both EA and non-EA orgs (notably including some higher level poisitons at the UN that had a multiple hour long work test as part od second and third rounds of the hiring process), although for the most part the EA ones have been compensated, and compensated fairly generously. I personally consider any uncompensated work test that takes more than about an hour to be a serious red flag for how I expect the org to treat overtime, work life balance and compensation.

I know that these positions are very competitive and that not engaging with them is not a choice for most people looking for a job, but I think that that might be the best way to disincentivize these practices- a drop in the quality and quantity of applicants while letting them know the reason why will probably be the most effective way to discourage this practice. I'm also generally a fan of naming and shaming for exploitative practices but that unfortunately tends to have an outsized negative effect on job seekers.

david_reinstein2y2

(Speaking in a personal capacity of course) I like the idea of using interviews and work tasks to help the organisation and help EA priorities etc. Consulting a variety of perspectives and backgrounds can be very useful, and otherwise we can end up wasting a lot of time, effort and insight in the job seeking and hiring process.

As as an interviewee I was sometimes frustrated when I put in a lot of work into a task I ~knew was not actually used/useful.

BUT I think these must be compensated and perhaps even generously compensated. Generously: to avoid the impression of being exploitative, and the temptation to do so without considering the value of the interviewee’s time.