Comments
What are good rubrics or rubric elements to evaluate and predict impact?
Together with Ozzie Gooen and Elizabeth Van Nostrand, I have recently been looking into rubrics to evaluate and predict the value of projects. These rubrics are each composed of factors like "scale", "tractability", "neglectedness", "willingness to pay", "past experience with similar projects", "scalability", "personal enthusiasm", and so on.
Some of these elements have a causal connection (e.g., personal enthusiasm means one is more likely to finish a project), but others are merely correlational proxies (e.g., willingness to pay doesn't directly cause higher impact). I'm interested in both types.
Otherwise, I have mostly been dividing rubric elements between those related to probability of success, those related to the magnitude of attainable impact, and those related to the amount of resources needed to achieve some impact. But presumably there are many other broad categories or sub-categories; I'd also be interested in those.
I've very recently started thinking about quite a similar question (focusing specifically on how to prioritise among possible research directions by predicting the likely direct and indirect impacts of the outputs).
So I'd also be very interested in people's thoughts on this question. And hopefully I'll have some thoughts to share in the coming weeks.
(By the way, was this post meant to be a "Question" post?)
Ach, yes.