Predictive analytics expert Ingo Mierswa, who also happens to be CEO and founder of RapidMiner, recently blogged about the relationship between big data
and smart data. The true power of big data, he explains, lies in creatively using advanced analytics to transform that 'big' data into 'smart' data and actionable business
Why Is Big Data Such a Big Deal?
First, Mierswa points out, big data can be defined in terms of the "3 Vs" that Gartner 's analysts have enumerated -- Volume, Velocity, and Variety. We're talking about really large volumes of data, and different kinds of data (some structured, some unstructured, etc.), possibly collected in batch mode, at intervals, and/or in real-time (that's the velocity component).
Volume, of course, is the most obvious characteristic of big data. The sheer volume of big data makes it hard to store and hard to retrieve. But Mierswa points out that its size is actually the least interesting quality of big data.
"The technology needed to store and retrieve this massive amount of data is coming along quickly, building on past developments like relational database systems and storage clusters. Now NoSQL databases and Hadoop are considered solid solutions for distributed data storage and retrieval." And, Mierswa predicts that many more companies will soon be using these technologies as underlying data delivery frameworks.
It's actually the variety of big data -- whether it's structured and/or unstructured -- that adds another layer of complication on top of the usual storage and retrieval challenges. One of the key challenges, Mierswa suggests, is organization. "Big data is fundamentally hard to organize." He points out that when you move beyond analyzing just structured data, you may, for example, "need text analytics to transform unstructured information like text, images, or videos into structured data."
Big Data Alone Has No Value
What? Did you read that correctly? Indeed, Mierswa writes, "Big data alone has no value! To extract value from it, you have to employ advanced analytics. You can only unlock the true power of big data when you learn how to transform it into smart data, data that helps you understand why the world works as we know it."
"It takes advanced analytical methods to find the correlations and underlying patterns that help us understand the present and predict the future," Mierswa explains. "If a company understands which customers will quit their contracts, or when a machine will break down, they can be proactive, solving problems before they occur and making decisions that will guard the company’s interests long-term."
What Can Big Data Do for You?
Ultimately, that's the million-dollar question. And, how can your business leverage your customer data to better serve your customers, while increasing sales and profitability?
"Everyone seems to be trying to figure out how best to tap into 'Big Data' to achieve their goals," Mierswa writes. The key, is recognizing that, "the true power of big data lies in creatively using advanced analytics to transform it into smart data."
Large-scale marketing and sales success requires looking for patterns, correlating customer actions and reactions, and testing different strategies to determine what works best. Software for data mining and predictive analytics, like that developed by RapidMiner, can provide the tools marketers need.
RapidMiner actually evolved out of the Artificial Intelligence Division of the University of Dortmund, Germany, under Mierswa's leadership. Over the past five years, the company has grown substantially by helping well-known customers such as Lufthansa, PayPal, Pepsi, Siemens, and Volkswagen by turning their big data into smart data and actionable business intelligence.