Surveying scientific experts seems like a promising way to gather informed opinions relevant for important decisions, and to improve common knowledge on important topics.
The IGM economic experts panel has been doing surveys for roughly 10 years, and I wonder how well both the surveyed consensus and the format of the platform hold up, as it might be a good idea to
- copy the general IGM platform to other scientific fields,
- offer ideas to improve the IGM panel, and more ambitiously to improve the economics consensus (e.g. by spotting biases), and
- use the survey results when they interact with topics relevant for EA.
Two ideas how to evaluate the survey's usefulness
- Look through IGM's survey questions and evaluate those which have something like a "correct" answer in retrospect. This requires somebody fairly informed about the surveyed issues.
- Interview someone from IGM, or a researcher knowledgeable about expert consensuses in economics or generally (suggestions?).
Two articles about the IGM survey
- Tyler Cowen evaluating some IGM polls in 2015: "But quite seriously, my opinion of the professional consensus — on topics outside an individual’s research specialty — really has gone down as a result of these polls." (link)
- discussion of biases in the economics elite, in the surveyed IGM questions and for which experts were selected (paper from 2016)
- IGM asking softball questions apparently aiming at eliciting textbook answers
- some schools of economics not being represented, e.g. the Austrian school
- clustering of responses according to political partisanship and departmental affiliations
Related questions that might be interesting
- what has been the impact of the IGM survey?
- did it influence policy makers? did it influence public discourse?
- does it have an impact on the future research topics of surveyed researchers?
- which fields are most promising candidates for a similar survey platform? are there already similar surveys in other fields?
- what influences the buy-in that the IGM experts panel enjoys from academics?
- what tradeoffs are involved in different formats of knowledge elicitation?