Several decades ago, researchers investigated how measured values of fundamental physical constants evolved over time. As expected, measurements gradually converged toward true values as instrumentation improved. However, this convergence revealed two unexpected patterns. First, estimated measurement errors were, on average, half the size of actual errors. Second, measurements were systematically biased toward values considered accurate at the time.
The most plausible explanation is not deliberate fraud but a natural human heuristic: when scientists obtain seemingly implausible results, they are more likely to suspect errors and adjust their calculations or experimental setup.
In empirical sciences, this consensus-seeking behavior slows but rarely prevents progress. Since scientists have no personal stake in whether a physical constant is higher or lower, their measurements—though pulled toward the current "Overton window" of acceptable values—still tend to fall closer to truth. Over time, the window itself shifts in the correct direction.
Policy research faces different incentives:
Consequently, instead of gradually converging toward correct answers, the Overton windows of acceptable results often drift toward socially popular viewpoints.
In long-term policy debates, empirical testing of proposed solutions is generally unfeasible. Determining whether a particular policy would help or harm humanity requires implementation, and even then—except for extreme outcomes (e.g., "AGI destroys mankind")—the results are likely to remain controversial.
However, while we cannot empirically test long-term policy recommendations, we may be able to test their authors. Consider what would happen if information about experts' biases—such as political biases or tendency toward overly optimistic or pessimistic predictions—were publicly available. In any major debate where numerous experts present recommendations, we could analyze how these recommendations correlate with different biases and use this information to select an optimal course of action.
Researchers' biases can be measured through forecasting-based empirical testing. Data from forecasting platforms demonstrate that people's biases clearly manifest in their tendency to systematically overestimate the likelihood of outcomes aligning with their personal preferences. For instance, data from Manifold Markets indicate that most users exhibit significant biases along several dimensions:
The data also reveal that, for markets not expected to resolve in the near future, users' biases have a far stronger impact on their betting choices than their intelligence or general expertise (estimated from average winning rates).
Thus, unless policy experts differ fundamentally from physicists and Manifold users, we should expect their research to be affected by personal biases to a similarly strong degree.
Two major obstacles prevent implementing this approach in practice:
If you have ideas on how to address these challenges, please share them in the comments.
