Some possible setups:
What are some reasonable approaches to estimating the expected values of such setups?
That's a great question! I don't have any good answer, but I've looked online and found some interesting papers so I'll just post some stuff I've got so far.
It seems like there is recently a shift toward "societal-impact focused research", as opposed to "quality-focused", driven mostly by the need to calculate Return On Investment. I think that this biases the current metrics/evaluators to be more short-termed and focused on health/security/tech-innovations.
Here, the authors ask research evaluators how they think about assessing societal impact. They have identified 5 dimension -
1. The Importance of the Underpinning Research in Evaluating Impact.
For more quality-focused evaluators, the importance of underpinning research when evaluating impact was driven by an underlying value system depicting a strong link between scientific and societal impact.
2. The Value of the Impact Versus the Value of the “Right” Impact.
For some evaluators, the necessity for research of a high quality to underpin societal impact was guided by the assumption that impact referred to ‘good impact’, as opposed to ‘negative’ societal impact.
3. Impact as Linear, Controllable or Serendipitous
A major underpinning factor influencing evaluators’ opinions was related to whether to view impact as related to ‘outside factors’ separate to the research, or something that was viewed rationally, therefore related to the quality of the research.
Towards the quality-focused extreme, evaluators envisaged a ‘pipeline’ from high quality research to societal impact – “a sort of translational pipeline is the okay term that tends to get used for taking a scientific discovery and pushing it towards some sort of laboratory test, new drug, or whatever, which, I guess, many people would view as some sort of impact”(P1OutImp5). Thus, the relationship between scientific and societal impact hinged upon the idea that “impact requires that you generate the evidence and then that you, in turn you get into guidelines and the people start using that information to change their practice”
4. Push Factors and Assessing Impact
Towards the quality-focused evaluator extreme, the assessment of societal impact was influenced by a belief that a researcher’s role in ensuring societal impact was limited solely to providing high quality research, whereas it was the responsibility of other, non-researchers to use this as evidence to pursue societal impact.
5. Measurable Impact Outcomes Versus Unmeasurable Impact Journeys
The final factor which influenced the evaluation scale was whether evaluators valued societal impact as a single, measureable outcome, or as a process or journey that, in many cases, is impossible to be measured.
Nice find! This seems like a useful step, though of course likely considerably different than what I imagine consequentialists would aim for.
Some low-effort thoughts (I am not an economist so I might be embarrassing myself!):
Thanks! This is interesting, will spend some time thinking about.
It's worth keeping in mind that it could actually be net negative if it pulls enough attention of other (potential) researchers away from better things to work on.
Agreed. I'd add that the EV of intellectual progress in certain fields could also be net negative for other reasons. What resonates with me most is concerns about increasing existential risk (e.g., because some dangerous technological development is accelerated; see also differential progress). But there could also be other downsides, such as risks of increasing meat consumption or environmental damage (e.g., via economic and population growth).
But this will vary from field to field, and based on many other factors, and there are of course many benefits to progress in many areas as well. I just raise this as a possibility/consideration.
Yea, I think there's a similar concern any time you make other fields more well run. That said, as a rule of thumb, this seems a lot safer than making many other fields less well run. It would be great to be able to apply intellectual abilities selectively, but when that's too hard, doing it generally seems fairly good to me.
Is it too naive / basic to just suggest value of information calculations?
Agreed, though the suggestions are appreciated!
VOI calculations in general seem like a good approach, but figuring out how to best apply them seems pretty tough.
But how do we estimate the EV of estimating the EV of general intellectual progress?
On a less facetious note, it's about the average effect of intellectual progress on innovation right? What EV comes from general intellectual progress that is not a result of innovation?
So you try to causally estimate the effect of innovation on things you value (e.g. GDP), and you try to create measures of general intellectual progress to see how those causally impact innovation. That's obviously easier said than done.
seems relevant, and I want to look into more deeply - Back to Basics: Basic Research Spillovers, Innovation Policy and Growth