The world produces more data now than at any point in history — by some estimates, over 2.5 quintillion bytes of data are created every single day. That’s a number that grows every time we search the Internet, post to social media, use GPS, walk into a store, swipe a credit card, or do most any other normal daily activity. Companies collect reams of such data every day: about their customers, employees, supply chains, and the world at large. But, what becomes of all those bits and bytes in this ocean of data? Often, not much — and that’s a distinct disadvantage for businesses looking for a competitive edge.
It’s estimated that about 73% of all existing data currently remains unexplored. This unanalyzed information is technically known as ‘dark data,’ and it represents a vast, untapped goldmine of potential understanding about our world. “It’s easy to collect data, it’s hard to turn it into insights,” says Jeremy Rader, GM of Enterprise Strategy & Solutions at Intel Corporation. “But, that’s the value that ultimately enables you to solve those big challenges for the company, your customers and partners.”
Looking more closely at our unused data can indeed yield astonishing results. In 1964, two radio astronomers researching distant galaxies noticed a mysterious noise pattern that kept appearing in their results. They examined it more closely — and what they found was the first significant evidence of the Big Bang that created our universe. Recognizing the value of data that other scientists might have dismissed as a nuisance won the astronomers the Nobel Prize.
For enterprise, harnessing unexplored data offers enormous opportunity. A team of technicians at Copenhagen Airport uncovered such an opportunity when they realized they could leverage existing data from passengers’ cellphones pinging the terminal’s Wi-Fi to create a real-time telemetry of travellers’ movements around the terminal. They used this insight to optimize passenger flows and minimize choke points at busy times. Using existing data, engineers were able to improve both the efficiency of the airport and their traveller’s overall satisfaction — an outcome any business would envy.
But as businesses begin to recognize the opportunity in dark data, there will be ever-increasing demands on the systems that store, process, and analyze data. To meet these needs, Intel is working with SAP to create a suite of hardware foundations and software solutions capable of handling the increased scale, capacity, and processing demands.
By engineering an environment where dark data can be leveraged, businesses can finally reap the value of their data, pull signal from noise, and uncover critical insights sooner. Great discoveries can happen when we focus a new lens on our existing data — who knows, there may be another Big Bang moment just waiting to happen.