From time to time, I come across the idea of using lotteries for all kinds of purposes, because they are an easy way to avoid bias. It seems that often the skills to get into a position are not the same as the skills needed to do well in that position and humans are bad at noticing this difference. For example, political campaigning probably needs very different skills than holding a political office (populist parties come to mind). In addition, it can easily increase diversity, as random chance cannot discriminate against people. Cases where lotteries seem to work well include:
- Democracy in Practice: A NGO in Bolivia that introduced a lottery for student government. Every student that would want to run for office is added into the pool of candidates. From that pool, all members of the student government are drawn randomly. This led to a higher diversity in the student governments and they tended to tackle projects that are more ambitious. Many students that would be really good at working in a student government might be too shy to do campaigning. Using lotteries gives them the chance to participate.
- Research grants: More and more research agencies realize that the peer review of research grants is flawed. While peer review seems to work reasonably well to detect bad research, it has problems in ranking good research and tends to err on the side of caution. This makes it hard for new ideas to be funded, even when they later turn out be brilliant (e.g., later Nobel Prize winners getting their grant rejected). To tackle this problem a modified lottery seems to work well. This means that there is a stage of peer review that rejects all proposals that have obvious flaws and all remaining proposals are added to the lottery.
- Citizens’ Assembly in Ireland: To enable a better citizen participation this assembly was established in 2016. It consists of 100 people and 66 of those were randomly selected citizens. Its purpose is to give recommendations on matters specified by the government. The assembly is provided with expert presentation and then discusses the topic in depth over several sessions and creates a report. Overall, the recommendations by the assembly were well received and it is seen as a good example to allow more direct participations of citizens without the dangers of populism.
My questions regarding this are:
- Does anyone know of some good objections to those approaches and to the use of lotteries overall?
- Are there EA organizations that use lotteries for research grants or even for things like applications? If so, what are their experiences with this approach?
- If there are no EA organizations that use lotteries: Why are lotteries not used more? Especially when it comes to grants, it seems like an approach that is easy to test, reduces work and might even give better results.
Its true that this is probably most suited to a funding scheme aimed at early researchers due to the limitations mentioned by you. However, I might think that the grant success might go up if you use a model were you sort out all bad research first, because your 20 % is probably relate to the overall number of applications. Or maybe you could give people more tickets in the lottery if they have proven they can produce good research. However, this might introduce new biases.
In addition, it might still be a good approach for intermediate researchers b... (read more)