«Peter Koen INTRODUCTION British Scientist, Francis Galton, in 1906 came upon a weight judging competition at an exhibition in Plymouth (Galton, ...»
PREDICTION, PREFERENCE AND IDEA MARKETS: HOW
CORPORATIONS CAN USE THE WISDOM OF THEIR EMPLOYEES
British Scientist, Francis Galton, in 1906 came upon a weight judging competition at an
exhibition in Plymouth (Galton, 1908). For sixpence you could wager a bet on the weight
of an ox and receive an award if your guess was closest to the actual weight. A total of 787 wagers were received from butchers and farmers, who presumably were experts, as well as clerks and others who had no expert knowledge. Galton ran a series of statistical tests on them and found that the crowd guessed that the weight of the ox was 1,197 pounds. The actual weight was 1,198 pounds. The crowds’ judgment was just about perfect. Francis Galton found that under the right conditions, groups are often smarter than the smartest people in the group.
The first public virtual stock market, which is known as the Iowa Electronic Market (IEM), a not-for-profit organization run by the University of Iowa, predicts the outcome of presidential elections. Berg et. al. (2008), based on an extensive analysis, found that IEM election predictions more than 100 days from the actual presidential election, in 1988, 1992, 1996, 2000 and 2004, were more accurate 76% of the times than popular opinion poll organizations such as ABC, CBS, CNN, Gallup, Harris or NBC. The traders were not even a representative sample of the voters as they were primarily students and faculty of the University of Iowa. Overall, these results show that the IEM markets may provide a more accurate long term forecasting tool than any polls.
Another well researched example (Spann and Skiera, 2003) is the Hollywood Stock Exchange (HSX, www.HSX.com). HSX issues MovieStock ahead of the actual release of the movie. The value of a share of stock represents the total of the US box office receipts, in millions of dollars, over the first four weekends after a movie has been released to greater than 650 screens. If a share of stock has a current value of $50, it implies that the box office receipts will be $50 million dollars during the first four weekends. Participants who think the share of the stock is undervalued (i.e. box office receipts will be more) buy shares. In contrast players who believe that the box office receipts will be lower will sell shares. HSX has more than 725,000 registered participants with an average of 15,000 individual visitors each day. There are no financial rewards.
However the participants with the most accurate forecasts are ranked and displayed creating an intrinsic motivation for them. The percentage error between the actual sales and the HSX predictions were 41% based on 140 movies (Spann and Skiera, 2003).
This is compared to expert predictions from Box Office Report and Box Office Mojo whose predictions for the same 140 movieswere 53% and 35% respectively. Thus the crowds were smarter the Box Office Report, but not as smart as Box Office Mojo. A more recent study by Karniouchina (2011) reveals inefficiencies in the HSX trader algorithm which would have further improved on the actual HSX predictions to the extent that they would have been better than the experts.
In 2004 James Surowiecki published a popular book, entitled “Wisdom of the Crowds,” to explain why large groups of people are often smarter than any individual. If the crowd is big enough and diverse enough you have access to so much more knowledge then you do if you ask one expert or even a small team of experts. This is apparent on the popular TV show “Who Wants to be a Millionaire?” When a contestant is stumped they can call an expert who is usually a smart friend or they can poll the audience. The experts gets the answers right about 2/3 of the time, but the audience gets the answer right 91% of the time.
Based on these promising results, many have begun using virtual stock markets for business forecasting (Cowgill, Wolfers and Zitzewitz, 2008). However, it is not obvious that success in large scale settings such as IEM or HSX will extend to corporations. The underlying theory is based on having a large volume of participants (Slamka, Skiera and Spann, 2013). In contrast, businesses require much smaller scale applications. In addition, incentives for trading are much different in corporations. HSX provides extrinsic bragging rights with accompanying social notoriety to the winners while iPredict (www.ipredict.co.nz), Betfair (www.betfair.com) and inTrade provide financial gain.
These same incentives cannot be used in corporations. This chapter evaluates the use of virtual stock markets in corporations. An overview of the three different types of virtual markets, prediction, preference and idea, used in
-2corporations are discussed in the next section. A conceptual model is then discussed in the subsequent section and is used as a reference for explaining why HZX, IEM and InTrade achieve high forecast accuracy and the conditions required by corporations to achieve similar results. The article concludes with a discussion of the process for setting up a virtual stock market in companies as well as guidance on how to select a virtual stock market platform.
VIRTUAL STOCK MARKETS IN CORPORATIONS: PREDICTION, PREFERENCE
AND IDEA MARKETSMuch of the nascent literature lump prediction, preference and idea markets together and call them prediction markets. However, how they are used in corporations, their outcomes and measurement accuracy are different. Prediction markets are defined as those which aggregate employees’ information to forecast a specific market event. For example, how many users will Gmail have at the end of a month, three months, six months? Other examples are predicting monthly sales of a product three to six months into the future. Prediction markets forecast a specific market event, such as sales, software release dates, etc., at some point in the future and have a real-world outcome which can be endogenously validated with 100% certainty.
Preference markets, which Dahan et. al (2011) also refers to as “Securities Trading of Concepts,” use the wisdom of crowds to predict product preferences of future customers. For example, a trader is asked to determine the desirability between eleven different air bike pump products by buying and selling securities in each of the different concepts. Their trading will reveal which concepts are preferred by the market. The difference between preference and prediction markets is that preference markets are focused on determining market preferences on a yet to be released product. In contrast, prediction markets forecast a future actual market event, such as actual product sales, using an actual completed product.
Idea markets represent a virtual platform where each idea is considered a stock in which employees can invest. Participants evaluate each idea and buy and sell the ideas that they regard as having the most value to the corporation. The share price of the idea provides management with the organization’s collective view. The outcome of the
A conceptual model which explains how virtual markets work is shown in figure 1 which provides a guide to understanding the factors that firms need to explicitly address in order to effectively use these tools. The model consists of 8 independent factors each of which will be discussed below, and is an adaption of the model originally developed by Kamp and Koen (2009).
Accuracy. The value of the virtual stock market to the corporation is related to its ability to accurately predict the future event. However, the measure of success is different for prediction, preference and idea markets. For prediction markets, the measure of success is the ability to predict the actual outcome. For example, how accurately can the market predict monthly sales for the new product three months in advance?
Measures of success for preference and idea markets are less straightforward. Dahan et. al. (2011) used multiple measures to determine the accuracy of preference markets including correlation with conjoint analysis, repeatability, and preference over surveys and actual sales. They found that preference markets where correlated with conjoint analysis, which is a statistical technique to determine how people value different product features, but not with actual future sales. This is not unexpected as preference markets evaluate the customers’ choices without respect to the product price, while in actuality, the buying decision is constrained by the customer’s budget. Accordingly, the accuracy measure for preference markets was the correlation with conjoint analysis.
In a similar manner, investigators developed multiple measures of success for idea markets including acceptance of the market by the corporation, senior management judgment of the quality of the idea, judgment of the idea by an expert panel, and senior management commitment to fund the idea to the next step (Soukhoroukova et. al. 2012).
Ultimately, the accuracy of the idea market is evaluated based on the senior management’s commitment to move the idea forward (Lauto et. al. 2013).
Accessible Information. Participants need a sufficient amount of information in order to make accurate predictions (Forsythe, et. al. 1999). For example, predicting the winner of the presidential election a full year prior to the actual voting will not be very accurate.
More information will be revealed as the actual voting day approaches.
Berg et. al (2003) found that the most accurate predictions of the 1988, 1992, 1996 and 2000 presidential elections occurred 31 days before the actual voting. More information increases the accuracy. A similar result was found in an extensive analysis of the HSX done by Karniouchina (2011). She found that prediction accuracy decreased when there were concurrent movie releases. The investors needed to combine large amounts of information from the concurrent offerings in order to understand the effects of competition.
What this means for firms is different for prediction, preference and idea markets. For prediction markets it means that sufficient information and specificity for the item being forecasted needs to be provided in the prediction campaign. For preference markets it means that product details including embedded attributes for each product, needs to be provided with sufficient clarity and specificity so that the traders can make clear choices. For idea markets it means that each of the ideas need to provide
-5an appropriate level of information so that the traders can judge the potential attractiveness and risk profile of the idea when compared to other ideas.
Truth-seeking Trading Behavior. A frequent criticism of virtual stock markets is that the stock price can presumably be manipulated by biased traders who are either ill-informed or are motivated to intentionally manipulate the value. However, the election markets, HSX and InTrade demonstrate that creating accurate markets is possible. Forsythe et.al.
(1999) explains why. Markets, such as IEM, are accurate based on the marginal trader hypothesis. Marginal or more experienced traders have more expertise in trading and are more knowledgeable about the market. The experienced trader drives the prices to the correct values and profits from the mistakes of the average or less informed trader.
For every dollar an ill-informed trader loses as a result of a mistake, the experienced trader gains a dollar. All traders need not be experienced traders, but there need to be enough to have a personal stake in the outcome to move the share price to its correct value.
What this means for firms is that it is desirable to increase the proficiency of all of the traders. Training should be provided to all users to increase the knowledge of participants into how the virtual stock market works.
Secondly, it is also important to ensure that there is a sufficient number of participants with adequate domain knowledge in the area of trading so that they can adjust for the mistakes of the less knowledgeable trader to assure that the share price moves to its correct value.
Domain Knowledge. The marginal trader hypothesis requires that experienced and knowledgeable traders participate in the market. This may seem obvious, but many of the virtual stock markets run in corporations do not properly incentivize traders with appropriate levels of domain knowledge from participating. Even with experienced traders, some inaccuracies are expected. Karniouchina (2011), in evaluating the accuracy of the HSX, found that movies with stars tend to be overvalued and thrillers are undervalued. Thus even movie buffs, with considerable domain knowledge, can be both under and over optimistic traders in the market. The dot.com bubble of 1997-2000 and the subprime mortgage recession in 2008 suggest that markets can always have
-6excessive volatility. The need for traders to have adequate levels of domain knowledge in the prediction, preference or idea markets in corporations represents a necessary factor needed for truth seeking behavior as indicated by the arrow in figure 1 linking these two variables.
What this means for firms is that some of the traders in the virtual stock market need to have adequate domain knowledge.
Trading Experience. While many employees are familiar with the traditional stock market few are familiar with virtual stock markets. James Surowiecki, the author of the Wisdom of the Crowds (2004), indicated in a McKinsey Quarterly article (Dye, 2008) that one of the shortcomings of predictions markets are “…that a lot of people inside organizations don’t find the market mechanism intuitive or easily understood. They find it very challenging to use, which limits the pool of people who participate…” (pg. 89). In the same article, Best Buy indicates that they continue to teach employees how to use prediction markets. Google found that new employees and inexperienced traders suffer from overpricing of favorites, optimism and extreme outcomes (Cowgill, et. al., 2008).
Accordingly, this variable will increase the truth seeking behavior of the traders as indicated by the arrow in figure 1.
What this means for firms is that some form of training is needed for all of the participants in the virtual stock market.