The automotive ecosystem is in the midst of a dramatic transformation: from total reliance on internal combustion engines to growing acceptance of electric and hybrid electric vehicles; from driven to driverless cars; from vehicle ownership to vehicle sharing and pay-for-usage ride-hailing — and everything in between. For the last two years, we have been studying this transformation using crowdsourced forecasting tournaments in collaboration with Good Judgment Inc. (GJ), an organization that excels in designing and implementing such tournaments, based on the pioneering research by Philip Tetlock and colleagues. Led by John Paul MacDuffie and Rahul Kapoor, its objective was to learn more about the likely trajectory of these emerging technologies by harnessing the wisdom of the crowd.
Much of the present research on forecasting tournaments is centered on efforts to predict economic and political outcomes. It is our continued belief that there is a real opportunity in exploring the use of this tournament design for the purpose of understanding the patterns of disruption in industries. We have been working with GJ since October of 2015 to design and host forecasting tournaments focusing on the emerging technologies and innovations within the automotive sector.
The Tournament Results
The first tournament ran from April 2016 to December 2016, and allowed us to get hands-on experience with respect to how these tournaments operate, and to assess their potential utility vis-à-vis industry forecasting. The second tournament ran from July 2017 to August 2018. This tournament built on the momentum that we generated during the 2016 challenge and was significantly broader than the previous challenge in terms of the topics covered. We have composed a report that details the design, the findings, and the learnings from the second tournament. Specifically, the report sets forth the following:
- An overview of the challenge
- A discussion on what we have learned about disruption of Electric Vehicles (EVs) and Autonomous Vehicles (AVs) from our challenges
- A discussion on the forecasting trends from the challenge
- A discussion about the top forecasters in our challenges
- A detailed analysis of each question in our challenge
- A deep dive into the critical EV-related topic of average battery cost
- An outline of our plans for the next challenge
This tournament attracted nearly 1,600 forecasters who made nearly 12,000 forecasts on predictions covering specific developments in technology, automaker strategies, the competitive landscape and the regulatory environment. This has allowed us to observe in real-time the uncertainties and the dynamics underlying this changing landscape, and to create a large-scale dataset involving tens of thousands of probabilistic predictions by forecasters on specific questions across a range of topics, along with the rationale for the specific prediction in many cases.
About Good Judgment Inc.
Good Judgment Inc. runs forecasting tournaments on a variety of topics by combining clusters of time-limited, specific questions to identify important long-term trends. It is the commercial spinoff of the Good Judgment Project, a successful research project run by Wharton professors Phillip Tetlock and Barb Mellers, demonstrating how forecasting tournaments could harness the wisdom of crowds. Most of the previous challenges have focused on geopolitical events; the disruptive vehicle forecasting challenge was Good Judgment’s first to focus on technology forecasting.
For more on Mellers and Tetlock’s research, and the initial Good Judgement Program, we recommend these stories in Wharton Magazine and Knowledge@Wharton, as well as Tetlock’s recent book (with Dan Gardner) Superforecasting: The Art and Science of Production.