Prediction Markets

An account of current online prediction markets by Brendan I. Koerner at Slate.

So far the validation of such market predictions seems unsystematic, anecdotal, and based on economisting theology. It is difficult to figure out a good research design that would assess the quality predictive information deriving from such betting.

Would the same claims for predictive accuracy be made for betting on horse racing? The stock market? Insurance? Gambling? When does participation in predictive markets differ from lotto–a stupidity tax? How do we get good comparison predictions made by other methods? In the face of no other information, the average of all independent predictions should in the long-run do well. When is this the case? How well? Under what conditions? How to make quantitative estimates? Are betting pools like taking the average?

Does it matter who participates at what level of investment? What about dependencies among the choices of the participants? Purely strategic betting? How is the prediction affected by the framing of the betting question, as will surely be the case? Should some predictions receive more weight than others in the aggregation, just as higher quality studies receive more weight in meta-analysis? Size of bet does not seem to be a good proxy for how informed the bet is; indeed there might be an inverse relationship! Can knowledge of the prediction affect the predicted outcome, as is often the case in human affairs?

How about a market predicting the conclusions of studies of prediction markets–want to bet on the findings of studies from the University of Chicago School of Business compared to those from a Department of Sociology in France? Or classical vs. behavioral economists?

Prediction markets seem a bit like factor analysis or data mining in statistics–techniques to try when you don’t have any good ideas.