I would advocate for controllably timed work tests whenever possible. Simply saying "please don't spend more than X hours on this work test" gives the opportunity to cheat by spending more time. Incentives for cheating are strong, because:
If you have enough applicants, some will cheat, and they will get a significant advantage. In rare cases, this may even deter people from applying. There was one position were I was planning to apply but then didn't because they had a non-controllably timed worktest (I don't want to cheat, somebody probably will cheat, and I am not super-well qualified for the position anyway so I would really need to shine in the work test -> not worth applying). (I admit that this deterrence probably doesn't happen often)
Great tools online for doing controllably timed work tests exist.
(I realize that it is not always possible to control the time limit, e.g. when the task is too long to be done in one sitting. I have no recommendation for what to do then, other than that I think Jonas Vollmer's comment in this thread seems reasonable).
My current view is to ask for both timed and untimed tests, and make the untimed tests very simple/short (such that you could complete it in 20 minutes if you had to and there's very little benefit to spending >2h on it).
I have quantitative data on that :-)
I asked 10 people for career advice in a semi-structured way (I sent them the same document listing my options and asked them to provide rankings). These were all people I would think rank somewhere between "one of the top cause prioritization experts in the world" to "really, really knowledgeable about EA and very smart".
6 out of 10 thought that research analyst for OpenPhil would be my best option. But after that, there was much less consensus on the second best option (among my remaining three top options). 3.5 people rated management at an EA organisation highest, 3 rated biosecurity highest, 3.5 rated an MSc in ML (with the aim of doing AI safety research) highest.
Of course, YOU were one of these ten people, so that might explain some of it :-).
I had many more informal discussions, and I didn't think there was strong consensus either.
(Let me know if you need more data, I have many spreadsheets full of analysis waiting for you ;-) )
Hey :-)
Regarding me being a bit of an outlier: Yes, I think so as well. I personally don't know anyone who applied for quite as many positions. I still don’t think I am a *very* special case. I also got several private messages in response to this post, of people saying they had made similar experiences.
Regarding compensation: I was lucky enough to have decent runway, so the financial aspect wasn't crucial for me, so I just forgot including it. I will edit that now.
Two additional possible reasons:
(PS: Your point 1 and 2 applied to me very much, but I didn't get the impression of points 3-5 being the case (I didn't think people consistently recommended EA orgs over other options))
I got feedback for most of the roles where I reached the last stage of the application process, usually not for others. Usually the feedback was that my application was good (that I would have passed the hiring bar), but that someone else was better. I also got more specific feedback, but that was different every time (which makes some sense, since the positions were different, the people evaluating me were different, and I tried to not repeat mistakes that I had previously gotten feedback on).
I think there are probably a few things that some EA orgs could improve and I hope to write a post about it soon. In the meantime, it might be useful to explain where some of these high numbers come from:
1. Un-timed work test (e.g. OpenPhil research analyst):
I think most EA orgs underestimate how much time a work test takes. Take for example the conversation notes test of OpenPhil's application procedure. In the email instructions to the test, you will find the following line: "Historically, we think people have spent 2-8 hours on this assignment. " But there is no indication of how much time you should/are allowed to spend. And since everyone knows that the process is really competitive, and your results keep on improving if you invest more time, many people invest a lot of time. I spent 16 hours on the task. I asked three other people how much time they had spent, and it was 8 h, 16 h, and 24 h.
2. Research proposals (e.g. FHI research scholar programme, OpenPhil biosecurity early career researcher grant):
Writing a research proposal just takes a lot of time. I spent 30 hours on my proposal for FHI. I know of 4 other people who applied. These are the times they spent on the proposal (full-time): one day, one week, one week, several weeks.
3. Trying to be really well prepared (my own fault, no one forced me to do that):
Knowing that the positions are competitive, I would often spend several hours preparing for (later-stage) interviews. E.g. when applying for the CEA local group specialist role, I spend 4-5 hours reading and thinking about CEA's strategy in movement building.
4. Travel time:
As stated in the post, I counted travel time at 50%. And Oxford is really far off :-)
---------
So depending on how exactly Wave's application process looks like, I might potentially have spent more than 10 hours on it as well :-)
(My answers might be very different to the ones that anonymousthrowaway might give).
Even if not a direct answer to your question, maybe it helps to illuminate the dynamics that contributed towards me making that many applications before moving on to other EA things. I personally did NOT think that jobs at EA organisations had clearly higher expected value than my other top options (which were working in biosecurity basically anywhere, or upskilling in machine learning via a master's degree). (There were very few exceptions, like Chief of Staff for Will or RA at OpenPhil, which I thought were outstandingly good).
Then why did I apply to so many positions?
1. I thought EA organisations were really talent-starved and needed me. I also thought that it would be easy to get a job. This is clearly my fault, and I think there would have been ways for me to reach a better knowledge of the situation. But I went to several EAGs, talked to dozens of people, read most of 80000 hours advice, and I think it was definitely quite easy to come away from these with the impression I had.
2. I got pretty far in a few application processes, so that encouraged me to continue applying.
3. Somehow, EA positions were the positions I heard about. These were the positions that 80000 hours emailed me about, that I got invitations to apply for, that I found on the websites I was visiting anyway, and so on. Of course, the bar for applying for such a position is lower than if you first have to find positions yourself. 80000 hours never emailed me about a position in biosecurity.
4. EA was the thing I knew most about. I knew a lot about the EA-sphere anyway. So it is easier to evaluate to which organisation to apply to, than say in biosecurity (which is a vast, scary field that I have very little knowledge about). If I apply to OpenPhil, that is definitely at least good. If I pick a random biosecurity organisation, I have to do my homework to figure out if it is promising or not.
5. Psychologically: My other top options (biosecurity anywhere, most likely not long-term-focused, or upskilling in ML) felt like "now I have to work really hard for some time, and maybe later I will be able to contribute to X-risk reduction". So still like I haven't quite made it. In contrast, working for a major EA org (in my imagination) felt like "Yes, I have made it now. I am doing super valuable, long-termism-relevant work".
6. Working at an EA organisation was the only thing that would contribute towards the long-termist agenda that I could hope to do RIGHT NOW. For the other things, I would need upskilling. So if discount rates on work are high, that makes EA orgs more attractive. (However, I don't believe that discount rates on "valuable work in general" are anywhere as high as the rates often cited in the EA-sphere, so that did not make me think that jobs at EA orgs are clearly better than my alternatives).
Finally, I think that the fact that I thought it would be easy to get hired by an EA organisation really is quite crucial. My points 3.,4.,5.,6. really mainly became relevant against this background. It's the difference between
Now, without clearly defining WHAT it even is that we want to improve, here are a few ways that I think this could be improved. (a rather lose connection of thoughts)
To improve 1):
To improve 3):
To improve 4):