Jorn Lyseggen
OUTSIDE INSIGHT
Navigating a World Drowning in Data
To Kai and Izzy
I won the cosmic lottery when you were born.
I am so proud of you. I love you so much.
UK | USA | Canada | Ireland | Australia
India | New Zealand | South Africa
Portfolio Penguin is part of the Penguin Random House group of companies whose addresses can be found at global.penguinrandomhouse.com.
First published 2017
Copyright © Jorn Lyseggen, 2017
The moral right of the author has been asserted
ISBN: 978-0-241-28164-2
Preface
Introduction
Part One: A New Digital Reality
1 Everyone Leaves Online Breadcrumbs
2 Mining Internal Data Is Looking at the Past
3 Mining External Data Is Looking into the Future
Part Two: A New Decision Paradigm
4 Outside Insight: A New Decision Paradigm for a New Digital Reality
5 The Value of External Data
6 The Value of Real Time
7 The Value of Benchmarking
Part Three: Outside Insight in Practice
8 Outside Insight for Boards and Executives
9 Outside Insight for Marketers
10 Outside Insight for Product Development
11 Outside Insight for Risk Management
12 Outside Insight for Investment Decisions
Part Four: The Future of Outside Insight
13 The Emergence of a New Software Category
14 Hard Problems To Solve
15 New Data Sources
16 The Potential Concerns of Outside Insight
17 The Future of Outside Insight
Appendix: Outside Insight Complimentary Licence
Sources
Acknowledgements
Follow Penguin
In the summer of 2001 I took a step back and tried to identify the macro trends that would shape the world in the years to come. I was set on starting a new company and tried to identify an important trend to ride. One trend that I found very intriguing was the explosion of online information. We were drowning in information, and manually it was simply too time-consuming and hard to make sense of it all. The obvious answer to the problem seemed to be a piece of software that could track and analyse all this information automatically.
That is how the company Meltwater was started. We were two guys and a coffee machine, and we started the company in a shack in Oslo, Norway (our address was Shack 15), with starting capital of US$15,000. Our vision was that when executives and decision-makers arrived for work in the morning and had their usual cup of coffee, our software would create a summary of what had happened in the world during the previous twenty-four hours in a form that was so simple to read that within seconds an executive would get the key updates that he or she needed about competitors, key clients and their own brand. Our slogan was ‘Informed decisions’, because we wanted to help executives to take advantage of the information that was now available online in addition to all the traditional data they had at their disposal.
At the time we did not realize how big a macro trend we had stumbled upon. Initially, we focused on online news and company websites. We didn’t anticipate the rise of social media. Facebook was founded in 2004, Twitter in 2006. Blogs didn’t reach mainstream attention until around 2007. All this new online content fuelled a need for new, sophisticated software with the ability to analyse billions of documents every day. Today smart algorithms, natural language processing (NLP), machine learning and big-data techniques are used to create analytics and insights from online news and social media that we would not have dreamed of back in 2001.
Over the years Meltwater has grown into a global business serving more than 25,000 corporate clients in more than a hundred countries. Our clients range from medium-size, local companies to large multinationals. In one way or another we serve 50 per cent of the Fortune 500 companies. We serve every industry, and our clients comprise everyone from Coca-Cola to the Vatican. From humble beginnings we have grown to 1,400 employees, with sixty offices in six continents.
From our clients we have learned ingenious ways to use external information. In addition to the obvious uses, such as competitive intelligence, measuring client happiness and product development, we have also come across a range of surprising cases that we hadn’t anticipated. The University of Oslo used our service to measure the speed with which a new spelling of the word ‘ketchup’ in Norwegian was adapted. A ten-person company from southern Sweden selling windows (glass ones – not the Microsoft software) tracked regional news reports for burglaries in order to generate sales leads. A government agency in a European country analysed online chat groups to investigate suspected cases of insider dealing.
Working with clients all over the world and seeing the value they can extract from external information has over the years created a growing realization that we have barely scratched the surface in terms of the role external information will play in decision-making.
Consumers and companies today produce online content at an unprecedented rate. Consumer engagement online and in social media is showing a steady increase. Companies are embracing the internet as a strategic arena to promote their brand, products and jobs, and are increasing their online investment and content creation accordingly. With the rise of new, richer content openly available on the internet, ever more sophisticated business insights can be created.
I believe we are on the cusp of a big transformation when it comes to corporate decision-making. I believe that in the coming years the use of online information will change the way boards are run, the way we develop strategies, the way a company’s health is measured and the way executive pay is earned.
These are mind-boggling prospects, and certainly way beyond anything I imagined when I started Meltwater back in 2001. Thanks to the World Wide Web and the rise of social media, the internet has become a treasure trove of consumer insights and competitive intelligence. Today this information is largely underutilized. As companies jockey to improve their positions in an increasingly dynamic and fast-paced competitive environment, the companies with the best ability to anticipate change, and to respond accordingly, will win. Central to this will be a company’s ability to use external data and to create insights from the outside.
Jorn Lyseggen
San Francisco, March 2017
On 25 April 2016, VentureBeat, a tech blog that covers the Silicon Valley ecosystem, reported that, in the week leading up to its Q1 financial report, Apple had been cutting all of its contract recruiters.1 This was taken as an ominous sign, and when Apple’s financial figures were reported the next day, they showed a 13 per cent drop in revenue.2 This was the first quarter in thirteen years that Apple had reported a negative sales development. The market’s reaction to the slump was to shave $58 billion off Apple’s market value, roughly equivalent to the market value of the German car company BMW.
This book describes the valuable insights that can be found in the information that companies and people leave behind on the internet – our ‘digital breadcrumbs’ – and how this information is mostly overlooked in corporate decision-making. Job postings, social media, blogs and patent applications are rich sources of forward-looking information. They reveal how much a company is investing, how happy its clients are and its future market positioning. In spite of their obvious strategic value, these sources of information are not commonly utilized today. In this book we will show that those who do pay attention to this new class of data types will develop a superior understanding of their competitive landscape and gain an ‘unfair’ advantage over their competitors.
This book tells the stories of many different organizations that are already using Outside Insight to gain a competitive advantage and improve their decision-making.
A special Facebook monitoring unit in the NYPD was able to use Facebook data to trace and convict the murderers of an innocent teenager caught in the crossfire of gang warfare, in a crime with no witnesses.
In its early days YouTube benchmarked its media coverage with that of its competitors to see how successful it was in building its brand and creating a following. Its early lead in media mentions was an early indicator that YouTube would be the winner in the online video space.
The American retailer Racetrac used external data to increase the accuracy of its revenue predictions. By incorporating external leading indicators – data normally not used for budgeting – it was able to reduce prediction errors by 15 per cent.
In just four years the Swedish watchmaker Daniel Wellington grew from nothing to selling more watches than Rolex, using Instagram as the primary marketing channel. Daniel Wellington mobilizes its clients as brand ambassadors and is an example of a new generation of brands that was ‘born social’ and fully masters the viral potential of today’s digital world.
Hike, a home-grown Indian messaging app, was able to overtake Facebook Messenger in less than three years and become the second most popular messaging app in India, beaten only by WhatsApp. Hike’s secret weapon is rigorous social media analytics, used to inform its product development. New product features are carefully determined based on consumer preferences found in social media.
EQT, a leading European venture capitalist, is developing a sophisticated data science tool called the Motherbrain to find early-stage companies with traction. This initiative is based on the idea that, as companies grow and become successful, they leave behind online breadcrumbs such as job postings, social media buzz and media coverage. By using sophisticated software to monitor the information that is generated online, EQT hopes that it can find the most promising start-ups in Europe before its competitors.
Social media can also be used to predict elections. In 2016 Meltwater correctly predicted the result of the Brexit referendum and the Trump victory in the US presidential election. Traditional surveys pointed, in both cases, to a different result, but online analysis painted a more accurate picture and showed that Brexit and Trump had dominated support in social media. When the votes were counted, the results were more in line with the social media analysis than with the traditional polls.
Most companies today do not utilize external data in a systematic manner but instead focus their analyses and rigour on internal data such as company financials. The problem with such an approach is that it is very reactive. Internal data is the end-result of historical events. Running a company based on internal data such as last quarter’s financials is like driving a car looking in the rear-view mirror.
The main thesis of this book is that decision-making is up for a major overhaul and needs to adjust to a new digital reality. The internet has transformed the way we communicate, get news, shop, socialize, advertise and bank. Yet, despite all these changes, corporate decision-making processes have remained surprisingly static.
In this book a new decision-making paradigm is proposed. It’s called Outside Insight (OI), and it is an approach focused on anticipating changes in the competitive landscape by following and analysing the online breadcrumbs that competitors, clients, suppliers and other players in your ecosystem leave behind online.
The new approach to decision-making proposed moves away from the old paradigm centred on inwardly focused Key Performance Indicators (KPIs), financials, annual plans and quarterly reviews. Rigour is instead directed towards real-time analyses of external data. It is an approach that moves the focus away from what you are doing to what the industry is doing and, crucially, to understanding and anticipating changes in your market conditions in real time.
The need to analyse the plethora of information found online will give rise to an entirely new software category. This new software category will be to external data what business intelligence (BI) is to internal data. In this book this emerging software category is also called Outside Insight (OI).
While BI is primarily focused on company-specific operational metrics, most of which are lagging performance indicators, OI is concerned with a real-time understanding of the ebb and flow of the competitive landscape in order to anticipate future threats and opportunities.
The technical requirements are very different for the two software categories. While BI software is mostly focused on data that is structured in nature, OI software needs to be much more sophisticated and will have to understand text and be able to find patterns in large quantities of unstructured data. For this reason OI relies heavily on techniques from big data, machine learning and predictive analytics.
OI is adding a new level of sophistication to the executive toolbox. With OI, executives can look beyond a company’s data and develop a real-time understanding of how the whole industry is developing. With the help of vast computing powers in the cloud and modern data science techniques, the impact of all external factors can be measured and analysed, bringing Porter’s Five Forces to life in dashboards and real-time alerts.
I wrote this book because I believe that Outside Insight will shape the next decades’ thinking around company strategy and decision-making. Outside Insight extracts valuable forward-looking insights from a whole range of data sets that are largely ignored today. This data is from a third party and is unaffected by internal biases. It can be used to create an apples-with-apples comparison of you and your competition – in real time. Such an analysis would create a lot of value for boards, executives and operational teams alike.
We are living in a world where we are drowning in data. So far the new data types online have been mostly ignored. We can choose to continue to ignore them and consider them all noise, or we can approach them opportunistically and mine them for new insights. By analysing external information found online we can derive new insights that we cannot find in our internal data. This Outside Insight will help us make better decisions and create more successful strategies. It is my belief that within the next few years most companies will have no choice but to invest in systems and processes to become Outside Insight companies in order to stay relevant.
Part I of this book, A New Digital Reality, describes how the world has changed and how new data types found on the internet can be mined for forward-looking insights.
Part II, A New Decision Paradigm, discusses how access to real-time information about your industry changes decision-making in three important ways.
Part III, Outside Insight in Practice, offers a simple framework for getting started with Outside Insight today and outlines how to incorporate it into more advanced stages down the road. Part III also includes practical examples of how Outside Insight is used in executive decision-making, in marketing, for product development, for risk detection and in making investment decisions.
Part IV, The Future of Outside Insight, outlines important technical obstacles that need to be solved, what new data types we can expect and what concerns Outside Insight can be expected to raise once it becomes more widespread.
Outside Insight is still in its infancy. We have a lot to learn before we can harvest its full potential. This book contains success stories of innovative companies that are already using Outside Insight to their advantage.
It is my hope that this book will inspire the reader to take advantage of external data in a more systematic way. It is also my hope that this book can be the first small step in the overhaul of corporate decision-making and start the process of adjusting to a new digital reality.