:format(webp)/cdn.vox-cdn.com/uploads/chorus_image/image/63698582/newnormal_youtube_800x800.0.1537226321.0.jpg)
Emerging technologies constantly redefine a “new normal” in our world today. Broadly accepted practices become obsolete as technology changes expectations and creates a new normal for how business operates and people work together. This fuels overall productivity and drives business results.
When I started my career, written communication meant printing a memo and distributing it in a physical mailbox. Email defined a new normal — informal, instantaneous and ubiquitous. Working on a computer required going into the office to use a terminal or desktop PC. Laptops and remote access created the new normal of work at home. Later, the combination of the Internet and email phones like BlackBerrys reset the new normal to work from anywhere.
Smart mobile devices and the cloud are redefining today’s new normal. From Uber to Salesforce, software services are transforming the consumer and business experience. Cloud services have also redefined what’s possible in business insight. Companies born in the cloud develop a deep understanding of their customer because they can analyze every detailed interaction to determine preference and behavior. That analysis of behavior moves insight to a new level — business understanding — where decisions are made based on all available data. This understanding enables a virtuous cycle of positive customer interaction, which in turn drives revenue and profitability.
More than 20 years ago, the desire to gain business insight from transactional data created the data warehouse industry. This desire for insight was also the force behind the emergence of “big data.” Big data builds on the idea that, in today’s world, machines (such as Web servers, manufacturing plants, medical equipment and sensors) are generating the vast majority of business data. This data is different from traditional structured transactional information — it lacks a fixed format, so it is often called “semi-structured.” If transactional structured data defines “what happened,” machine-generated, semi-structured data explains “how it happened.”
Not surprisingly, there are important relationships between the “what” in structured data and the “how” in semi-structured data. The ability to bring together the “what” and the “how” is the key to moving beyond business insight. Analyzed together, this information can create business understanding.
But this is easier said than done. Even large companies with considerable resources struggle to attain business understanding because of the complexity of current solutions. While the need is apparent, achieving business understanding from all data is by no means the new normal.
Why not? Today, structured data is analyzed in expensive, special-purpose data warehouses. These systems are poorly suited to handle the dynamic data models required by semi-structured data. To work with semi-structured data, some customers are installing massively parallel clusters based on Hadoop. Both traditional data warehouses and Hadoop clusters are complex infrastructure which require special knowledge to set up and are costly to maintain. Even with all this infrastructure, there is no single solution that brings together structured and semi-structured data into a single solution.
The problem with these systems is that they were born in a different era. They were designed for tasks that made sense in the past, but the business needs have changed. Attempts to solve this problem with existing technology have proven to be complex, expensive and fragile. While they provide some level of business insight, they will not define a new normal of business understanding.
So, how can we apply the cloud to this challenge? To begin, it is important to understand that while it is possible to host existing applications in cloud infrastructure, this does not make them cloud applications. A cloud application is a software service. It enables connectivity from anywhere, on any device. Cloud applications are self-managed. They are elastic, meaning that data and computing resources grow and shrink based on user needs. Cloud applications generate semi-structured data that reveals usage patterns. The cloud opens the door to bringing together structured and semi-structured data into business understanding.
If you are looking for business understanding, you won’t find it by hosting a traditional data warehouse or Hadoop cluster on cloud infrastructure. Because these systems were born in the past, they don’t have the fundamental attributes of the cloud: They aren’t self-managing services, they aren’t elastic, and neither provides a solution that brings together structured and semi-structured data.
If you want a business understanding solution for today’s world, look to the cloud and reimagine the data warehouse:
Service, not infrastructure: Accessible from anywhere on any device, fully self-managed and self-tuning. Create an account, load your data and run queries. No infrastructure to worry about. No partitioning of data or tuning of any kind. Focus on gaining business understanding, not the infrastructure required to make it happen.
Elastic: Grow or shrink data, the compute capacity used to run queries and users on demand. Data storage and compute are cost-effective. Only pay for the resources used — not some fixed infrastructure purchased ahead of time. Create a virtual data warehouse on demand — and then make it go away when you are done. If a complex query is taking too long, make the virtual warehouse bigger. Then shrink it when you’re done.
One system for all business data: A single relational solution that uses SQL and combines both structured- and semi-structured data. The tools you use today — from Informatica to Tableau and Excel — just work. Business understanding is achieved.
Business insight has long been overdue for a technology shake-up. When data warehousing is reimagined for the cloud, it will create a new normal of business insight — business understanding. When that happens, business will never be the same.
Bob Muglia, CEO of Snowflake, has more than 20 years of management and leadership experience. At Microsoft, he led several business groups, including Developer Tools, Servers, Office and Online Services. Most recently, he was president of Microsoft’s $16 billion Server and Tools Business, responsible for products such as Windows Server, SQL Server, System Center and Windows Azure. Following Microsoft, he was EVP of Software and Solutions at Juniper Networks.
This article originally appeared on Recode.net.
