I will say this upfront, so you can weight my post however you want, but I applied to a researcher role at Givewell and got through to the final round before being rejected. I have substantial experience of hiring dozens of people at all levels of seniority and at every stage of the recruitment process.
Givewell is the most unprofessional and lackadaisical organisation with which I have ever dealt. An application process that should have taken 6-8 weeks at most took more than five months. Not least because it took them more than two months to make an initial sift of my application and get back to me. This initial sift really should not take more than a week, given the small amount of information that the application form requires.
Although the first written exercise seemed to run effectively (a response time of one week - perhaps I just got lucky there), things fell apart again at the second round. For example, the two Givewell employees that joined the call I was allowed to schedule before submitting my second round exercise seemed completely disinterested and uninformed regarding the assessment exercise.
Moreover, I would have thought that an effective second round would at least incorporate another call after submitting the exercise to discuss the submitted work and go over various different factors that were (or were not) included. That would provide information on how the applicant works and engages with colleagues, which is a crucial component in any workplace. However, Givewell’s application process assumes that people work entirely in isolation.
This difference from how good workplaces actually function is also reflected in the fact that they do not provide any information regarding what they actually want in the assessment exercises. There is no instruction to e.g. cover issue x, or focus on area y, despite the fact that in a real workplace those instructions would be provided (or at least discussed before the exercise).
Hence, despite Givewell's claims that the exercises reflect work tasks, that clearly is not the case as they do not reflect how work is actually conducted in a well-functioning organisation. (And if it reflects how work is conducted in Givewell, then that just proves that Givewell is not a well-functioning organisation.)
Givewell then took 50% more time than they said they would to get back to me regarding my submission for the second round application. And the response they provided was just a boilerplate rejection email the day before a national holiday. Said boilerplate rejection email claimed that I was “not a good fit for the role” - either that is true and they should have realised that at the initial sift stage rather than make someone who was not a good fit go through the application process, or they really need to change the rejection letter email. Either way, it’s not exactly a sign of competence on Givewell’s part.
Moreover, from what I can gather, the application process seems to discard all information previously obtained at earlier application stages. For example, the second stage of the process appears to use only the second stage exercise, rather than take into account the first stage exercise and the application form. This is clearly inefficient as it ignores information that is freely available. If Givewell wishes to be taken seriously, then it should use all information from previous application stages rather than just using the specific exercise submission at each stage.
Even worse is that despite my getting through to the final round, Givewell refused to provide any feedback on what I could have done to improve - I can understand such an approach after the initial sift or even after the first round (due to the sheer number of applicants at those stages), but it is common courtesy and accepted good-hiring practice to provide feedback to final round applicants (who are not going to be that numerous so providing feedback at that stage would not be burdensome). Not providing feedback at this stage is both rude and lazy.
For an organisation that is proud of its (supposed) transparency, the hiring process is not transparent at all: the lack of information regarding what they’re looking for at each stage; the absence of any criteria given as to what they will judge the exercise on; and the refusal to provide any feedback are each the very opposite of transparency. Even an organisation as opaque as the Civil Service provides an exact set of criteria against which an application will be judged, and provides feedback once an application has been reviewed. The fact that Givewell is less transparent than the Civil Service is a damning indictment.
If Givewell was serious about welcoming outsiders’ input into what they could do better, they’d work with experts to improve their hiring process. But they’re not a serious organisation, so I suspect they’ll ignore this.
On the whole, Givewell's lack of effort and competence in the hiring process really makes me doubt the quality and accuracy of their work assessing the cost effectiveness of other charities. If they cannot be bothered to do a good job in the hiring process, they also cannot be bothered to do a good job in examining a charity or intervention.
I don't think CEA should share specific criteria. I think they should give rejects brief, tentative suggestions of how to develop as an EA in ways that will strengthen their application next time. Growth mindset over fixed mindset. Even a completely generic "maybe you should get 80K advising" message for every reject would go a long way.
Earlier in this thread, I claimed that senior EAs put very little trust in junior EAs. The Goodharting discussion illustrates that well. The assumption is that if feedback is given, junior EAs will cynically game the system instead of using the feedback to grow in good faith. I'm sure a few junior EAs will cynically game the system, but if the "cynical system-gaming" people outweigh the "good faith career growth" people, we have much bigger problems than feedback. (And such an imbalance seems implausible in a movement focused on altruism.)
I'd argue that lack of feedback actually invites cynical system-gaming, because you're not giving people anywhere productive to direct their energies. And operating in a low-trust regime invites cynicism in general.
Make it clear you won't go back and forth this way.
This post explains why giving feedback is so important. If 5 minutes of feedback makes the difference for a reject getting bummed out and leaving the EA movement, it could be well worthwhile. My intuition is that this happens quite a bit, and CEA just isn't tracking it.
Re: making a stink -- the person who's made the biggest stink in EA history is probably Émile P. Torres. If you read the linked post, he seems to be in a cycle of: getting rejected, developing mental health issues from that, misbehaving due to mental health issues, then experiencing further rejections. (Again I refer you to the "Cost of Rejection" post -- mental health issues from rejection seem common, and lack of feedback is a big factor. As you might've guessed by this point, I was rejected for some EA stuff, and the mental health impact was much larger and longer than I would've predicted in advance.)
I think we would prefer that rejects make a stink to management vs making a stink on social media. And 5 minutes of feedback to prevent someone from entering the same cycle Torres is in seems well worthwhile.
Again, I do have significant knowledge related to giving feedback at scale. It isn't nearly as hard as people say if you do it the right way.
This seems like a red herring? I assume anyone applying for an analyst position at Givewell would be applying for a similar type of position at the Civil Service. White collar work may be hard to quantize, but that doesn't mean job performance can't be evaluated. And I don't see what evaluation of on-the-job performance has to do with our discussion.