This list of forecasting organizations includes:
- A brief description of each organization
- A monetary estimate of value. This can serve as a rough but hard-to-fake proxy of value. Sometimes this is a flow (e.g., budget per year), and sometimes this is an estimate of total value (e.g., valuation).
- A more subjective, rough, and verbal estimate of how much value the organization produces.
This started as a breadth first evaluation of the forecasting system, and to some extent it still is, i.e., it might be useful to get a rough sense of the ecosystem as a whole. After some discussion on whether very rough evaluations are worth it (a), people who prefer their evaluations to have a high threshold of quality and polish might want to either ignore this post or just pay attention to the monetary estimates.
Note that the monetary value column has different types of estimates.
|Flutter Entertainment||~$20B (~£18B) market cap||Gambling|
|Kalshi||~$30M in VC funding||Prediction market|
|Metaculus||~$6M in grants||Forecasting site|
|Polymarket||~$4M in VC funding||Prediction market|
|Cultivate Labs||$5M to $80M (estimated valuation)||Forecasting platform as a service|
|Good Judgment||$3M to $50M (estimated valuation)||Forecasting consulting|
|Manifold Markets||~$2M in early stage funding||Play money prediction market|
|Insight Prediction||—||Prediction market|
|Tetlock’s research group||$1M to $20M (estimated grant flow)||Research group|
|PredictIt||$0.5M to $5M (estimated value)||Prediction market|
|Epoch||$2M in grants||Research group on AI progress|
|Swift Centre||$2M in grants||Forecasts as a public good|
|Quantified Uncertainty Research Institute||$780k in grants||Software and forecasting research|
|Samotsvety Forecasting||$200k to $5M (estimated valuation)||Forecasts as a public good, forecasting consulting|
|Czech Priorities||—||Institutional decision-making using forecasting|
|Hedgehog Markets||$3.5M in VC funding||Prediction market (crypto)|
|INFER||Uncertain. Estimated $2M/year in grants||Forecasting platform|
|Nathan Young||$180k in grants||Forecasting question creation|
|Sage||$700k in grants||Forecasting research|
|Global Guessing||$330k in grants||Forecasting journalism|
|Social Science Prediction Platform||At least $838k in grants||Forecasting in an academic context|
|Augur||$60M crypto market cap||Prediction market|
|Confido||$190k in grants||Forecasting tooling|
|Hypermind||—||Play money prediction market|
|Replication Markets||$150k in forecaster rewards||Forecasting experiment|
|Gnosis||$230M crypto market cap||Smart contracts, including for prediction markets|
Monetary value (market cap): It has a ~$20B (~£18B) market cap.
Social value: Betfair and Paddy Power have some markets with public utilities, e.g., around elections, but focus mostly on sports. However, its social value is otherwise low or negative—e.g., by enabling or creating gambling addicts.
Note: There are many other betting companies, e.g., Draft Kings, which I will not review here because they tend to focus mostly on sports, though I will note that they tend to have a high market capitalization.
Monetary value (VC funding, valuation): Kalshi (a) has received ~$30M in VC funding. My sense is that it’s probably worth much more than that ($100M+) due to being the first prediction market regulated by the CFTC. Previous markets—like PredictIt or Iowa Electronic Markets—were only operating with a no-action letter.
Social value: There has been a bit of drama around Kalshi recently. In recent times, the CFTC withdrew its letter of no action from PredictIt, and fined and banned Polymarket from operating in the US. These are two of Kalshi’s competitors, and my sense is that Kalshi probably contributed to these events. As such, I’d estimate Kalshi’s impact so far to probably be negative, on account of these anti-competitive practices. In the future, however, if Kalshi contributes to the adoption of prediction markets in the US, and this leads to better decision-making, it could end up having a large positive value. It is also possible that their contracts have value as hedging instruments.
What it is: Metaculus is a forecasting site and a public benefit corporation that spearheads forecasting initiatives.
Monetary value (grant flow): Recently, Open Philanthropy granted them $5.5M (a). With a 5% discount rate, and asuming the $5.5M grant is for 2 years will be indefinitely renewed, this makes1 Metaculus' discounted cashflow be worth $55M. Metaculus previously received $308k (a) from the EA infrastructure Fund in 2021 and $65k (a) from the EA Long-term Future Fund in 2020.
Social value: With its $5.5M Open Philanthropy grant, Metaculus has been growing its already 12-person-strong team (a), and is hiring for a number of positions (a). So it’s one of the EA forecasting organizations most able to scale and deploy human capital in terms of sheer number. The community which makes up its forecasting site is also reasonably strong.
What it is: Polymarket is a crypto prediction market. It combined various technologies: magic.link, Polygon (a) and the Gnosis conditional contract in order to build a usable crypto prediction market at a time where the previous iteration, Augur, had become unusable in practice due to high fees on the Ethereum blockchain.
Social value: Polymarket has had a range of public interest markets, on politics, COVID infection numbers, or the Russian invasion of Ukraine.
What it is: Cultivate Labs is a company which offers infrastructure for forecasting tournaments. They currently host Good Judgment Open, INFER, the Cosmic Bazaar, and a Czech Prediction platform—and probably a few more platforms that I don’t know of.
Monetary value (valuation): I’m pretty uncertain about how much they are worth, and I’d give a guess of $5M to $80M.
Social value: I used to view them as a bit clunky and outdated, but I’ve come to respect that they are one of the only few groups willing to offer a full package, with support for multiple years.
Monetary value (valuation): I’m fairly uncertain how much they would be worth. Maybe a very wide interval would be $3M to $50M.
Social value: Good Judgment has helped build awareness of superforecasting practices. Their forecasts provided to their subscribers have probably influenced the decisions of their readers and of those who commissioned them. Their training at government and corporations probably marginally increased the decision quality of clients, but it’s unclear to me to what extent and for how long.
What it is: Manifold Market is a play-money prediction market site with a really nice user experience.
Monetary value (VC funding, valuation): Manifold Market has received $2M in funding, ($2M in seed funding (a), of which $1M from the FTX Future Fund’s regrantor program (a)), at a $15M valuation. They have a superb software engineering team, and one of the fastest development speeds, but on the other hand their user acquisition has been a bit slow (a). My sense is that their valuation would be somewhere between $10M and $50M
Social value: So far, by making an eminently usable site, they allow users to signal their beliefs and challenge others to do the same. They have also hosted a few tournaments, and at least the Clearer Thinking tournament (a) was probably decision-relevant. But their value is linked to their future growth.
Monetary value (valuation): This would depend on their usage numbers, which aren’t public. I’d say it’s worth at least $5M, but whether the upper bound is $50M or $500M probably depends on the team’s execution over the coming year.
Social value: It is one of the few prediction sites which hosts real-money somewhat liquid markets on the Ukraine-Russia conflict.
Tetlock’s research group
Monetary value (valuation, grant flow): They have received at least $2M from Open Philanthropy (1 (a), 2 (a)), and probably more than that through the University of Pennsylvania. It’s unclear to me how much to value a research group, but I’d give a ballpark of $1M to $20M.
Social value: Tetlock’s work on expert political judgment and on superforecasting was seminal work that kick-started the current forecasting community. In recent times, it seems likely that work stemming from their work on estimating existential risk (a) could be action-guiding for large organizations and governmental bodies, but most of the output from that research direction hasn’t been published yet.
What it is: PredictIt is a US-based prediction market. Historically, a community of sharp prediction market players coalesced around it. But in recent times, the CFTC has withdrawn its permission for it to operate in the US.
Social value: PredictIt was fairly instrumental in the creation of a passionate if sometimes a bit cut-throat prediction market community. Its future value depends on its survival.
Social value: My sense is that their research has usefully informed some OpenPhilanthropy decisions. Open Philanthropy donated around $80M in 2021 to the “Potential Risks from Advanced AI” cause area. A 5% improvement in that decision-making would imply a value of $4M/year.
Swift Centre For Applied Forecasting
Social value: Their forecasts (a) lead to better decisions. My sense is that the organization is still getting up to speed, and that when running at full operational capacity, they’ll be producing many more predictions.
Quantified Uncertainty Research Institute (QURI)
What it is: QURI is a “new initiative to advance forecasting and epistemics to improve the long-term future of humanity. We write research and make software.” In particular, we are developing Squiggle, a web-capable programming language for better estimation, and doing research around scalable forecasting and estimation.
Social value: Most of QURI’s value is coming from the value of Squiggle (a), which people are now starting to use for cost-effectiveness estimates, and from the value of my own research (a), which might move the Effective Altruism movement more in the direction of using quantified estimates even for speculative stuff.
Note: This is the organization that I work for.
Monetary value: My sense is that we could find some funder to sacrifice $200k to $5M if our group’s existence depended on it. But I haven’t actually tried that!
Social value: We’ve published a variety of public forecasts, and we find ourselves more and more to be a part of Effective Altruism’s general epistemic infrastructure.
Monetary value: I couldn’t find grantmaking information from a quick online search.
Social value: It’s possible that they could find generalizable lessons about using forecasting to influence policy and improve decision-making that could then be extended to other countries. But this is not certain.
Monetary value: They raised $3.5M in funding back in the crypto bull days.
Social value: They have been particularly innovative around no-loss markets, and peer-to-peer markets. But value depends on future usage volume.
What it is: INFER is a forecasting platform which seeks to influence US government policy around transformative technologies.
Monetary value: INFER is hosted at ARLIS (a), at the University of Maryland, which received an ~$8M grant from Open Philanthropy specifically for forecasting (a) (over two years, over two platforms), which I find very confusing because I don’t think they have deployed much of that capital yet.
Social value: INFER aims to influence decisions by the US government around technology policy. To the extent that this is the case, it could be valuable indeed, but this is hard to evaluate from the outside.
Smaller forecasting entities
Nathan Young is a smallish forecasting influencer followed by 6k people on Twitter. He recently got $182k from the FTX Future Fund regranting program (a) to build a platform for forecasting question generation, an alpha version of which can be seen here (a).
Sage is an organization dedicated to forecasting tooling and research. It got $700k from the FTX Future Fund, and has built the tools on quantifiedintuitions.org (a). But it hasn’t deployed most of its capital yet.
Global Guessing (a) is a forecasting site which made and gathered forecasts early on the Russian invasion of Ukraine, but which seems pretty dormant since then. They have received ~$330k from the FTX Future Fund. (a)
Social Science Prediction Platform
The Social Science Prediction Platform (a) collects forecasts from academics on upcoming papers. This can have a range of public benefits, such as estimates of how surprising results in the social sciences are. Personally I’d be hopeful about the ability to construct a more legible or objective Bayesian prior. I could quickly find that they have received $346k (a) from the Survival and Flourishing Fund, and an additional $492k from the FTX Future Fund's regranting program (a)
Replication Markets (a) was a super-interesting experiment in having forecasters predict the outcome of replications. It has now ended, and it’s uncertain whether the creators will follow up with something as interesting. They had ~$150k in forecaster rewards.
Gnosis is much like Augur, but less successful. They also have a GnosisDAO (a). The implied market cap of their token is worth $230M (a), based on the Gnosis safe (a). But it doesn’t do much forecasting these days.
Thoughts on this ranking
On the money flows
Money flows are interesting, but flawed as a measure of value. The spending rate or VC money raised does not correspond to the value a project creates. The high valuations for crypto projects could be mostly illusory, and for the case of Gnosis and Augur, they probably are.
Still, a monetary ranking has some advantages
- There may be some ambiguity to it, but it allows one to differentiate between <$500k projects, $500k to $5M projects, and $5M to $50M projects.
- It has an honesty to it, in that money is a somewhat hard-to-fake signal
- It is a fast proxy to estimate
On the estimate of social value
The estimate of social value isn’t a quantified estimate, but rather a few lines of observations, and maybe a path to impact. I think I could translate this into a numerical relative value scale, but it would take a fair amount of time.
On the different pathways to impact
I’m seeing mainly three pathways of impact for forecasting projects:
- Providing consumer value, e.g., betting markets or hedging utilities that people want to use
- Providing informative markets about events of interest, which can apply pressure to decision-makers inside governments and corporations to make better decisions
- Providing tooling that people making decisions can make and incorporate as part of their decision-making
I’m mostly bullish on the third option, but this deserves more elaboration.
On how this estimate could be improved
This estimate could be improved by having numerical estiamates of impact. This would allow for comparisons around efficiency &c. It could also be presented better, i.e., in a table, or more creatively, in a map.
Significance given FTX's current troubles
Given FTX's current troubles, it's very possible that there will be less money floating around for altruistic forecasting projects. At the same time, some of the projects FTX has donated to may prove successful, and other funders may want to step in.