Big Data: What is It & How Can It be Used?

The New Face of Market Insight

The design of remarkable customer experiences begins with customer and marketplace insights. It always has and always will. Viral videos, memorable taglines, and actively used apps all start with a solid understanding of the person and the environment for which they are designed. It is logical: The more insight we have into our prospects and customers, the better we can meet their needs; the better we understand what is going on in the marketplace, the faster and more appropriately we can act. What is changing is the underlying data with upon which our decisions are based, the analytics we use to find meaning in the data, and the type of customer experience we are able to deliver as a result.

In the past, market research was derived from customer surveys, panel data, third-party transaction data, and often siloed customer databases, which were supplemented with qualitative data. We made key business decisions based upon what we would now consider limited information, years of experience, and a good amount of intuition. Today we are gathering data from seemingly everywhere: e-mails, texts, searches, product reviews, recommendations, customer service records, and more. This explosion of new types of information is known as big data.

Big data has captured the attention of organizations and institutions across multiple industries. Being able to investigate large datasets is intriguing because it holds the promise of predicting behavior with much more accuracy than is possible using smaller volumes. We can also integrate datasets to create synthetic data, data that cannot be obtained by direct measurement, from which we can often obtain deeper insight.

Until recently, big data analysis was theoretical at best because datasets were too complex, large, and cumbersome to capture, manipulate, analyze, and access using conventional databases. Developments in cloud-storage capacity and the ability to process information efficiently using tools like Hadoop, which allows companies to distribute computing tasks across servers to simplify data, are critical innovations that are making it possible for us to gain business value from the reams of data we are capturing.

Although we are still in the early stages, it is clear that organizations that are able to master the big-data and analytics opportunity will reduce their time to insight and increase the relevance of their interactions significantly. Excited by the potential, companies are investing substantial time and funds into capturing, managing, analyzing, and interpreting these new forms and combinations of data. Marketing initiatives are driving much of this activity. The goal, of course, is not to just purchase technology, but to be able to find the big insight within the data, and to apply that insight in concrete ways to deliver remarkable customer experiences.

What Exactly Is Big Data?

A clear definition of big data has yet to emerge; most definitions are descriptive. One of the properties of big data is its size, hence, its name. To get a sense of the magnitude of big data, consider the context provided by Eric Schmidt, Google’s former chief executive officer: “Between the birth of the world and 2003 there were five exabytes of information created. We [now] create five every two days.”2

From day 1 to 2003 there were 5 exabytes of info created. We now create 5 every 2 days Click To Tweet

Without even knowing what or how big an exabyte is, it is clear that the amount of data that is out there has grown significantly.

exabyte and zetabyte definition

How Big is an Exabyte?

What is fueling this growth in data? We are. As we go about our days, each of us leaves a digital footprint. Our individual and collective footprints are growing, as more devices and sources are added. If we pay by credit card for our gas or groceries, we generate transaction data that is collected by companies. If we are part of a loyalty program, every time we purchase anything, even if it is by cash, our purchases are recorded and captured by companies’ data systems. If we visit a given website, our activity—what we viewed, and for how long, and what search terms we may have used to arrive there—is captured. If we like a photo on Instagram, update our status on Facebook, tweet, or share our location through our mobile phones, it is recorded. If we sign on to any of these sites or apps via Facebook or Twitter, even more data is shared.

A plethora of data is also being generated from the Internet of things—smart devices embedded in objects—and from traditional secondary sources such as census data, macroeconomic data, psychographic data, weather patterns, and Dun and Bradstreet and Experian reports. It is not surprising that as marketers we often feel as if we are drowning in data.

Another characteristic of big data is its complexity. For years now, companies have been analyzing what is known as structured data. Transaction records like those we generate when we use a credit card or a loyalty card are considered structured because the data fits into traditional numerical-based database-management programs. Today over 80 percent of data is considered unstructured; our clicks, texts, searches, book reviews on Amazon, and blog postings are more complicated in form and cannot be stored and managed in traditional ways. To take full advantage of big data requires us to be able to combine structured and unstructured data and analyze these enhanced data streams. To do so we must be able to transform unstructured data into structured numerical data.

Speed is also a property of big data. Continual updates to data— think how often our phone-location data changes—and the frequent introduction of entirely new data sources, such as the latest mobile applications, add an additional layer of complexity to the puzzle. Our ability to extract insight from this data and act on it, in some cases in real time, can be what makes it possible for us to see an opportunity before our competition, make use of data before its shelf-life has expired, and provide remarkable experiences for our customers.

Where Is the Insight?

Big datasets in and of themselves do not have much value. The ability to analyze data, find the signals amidst the noise, and apply those insights to business decisions quickly is what makes them useful. Without reliable analytics, big data is a big nada. Fortunately business analytics are evolving to keep up with big data’s multiple new forms and sources.

Without reliable analytics, big data is a big nada. Click To Tweet

Dr. Michael Wu, principal scientist at Lithium Technologies, which develops customer experience software, provides a helpful overview of the three types of business analytics—descriptive, predictive, and prescriptive—in his blog,The Science of Social.3 According to Wu, the majority of business analytics used by companies today are descriptive; derived from operational data, they report what has taken place. Descriptive analytics answers questions like, “How many page views, shares, checks-ins, and replies to blog posts have we received?” This highly useful data that helps us evaluate the effectiveness of our marketing efforts. Filters can be applied to consider data from different perspectives and discover important influences. For example, a geo-filter allows us to consider how many people in a specific location shared our content. A weather filter allows us to explore the impact of weather conditions: How did a week of heavy downpours impact sales. While descriptive statistics are informative, Wu cautions they do not explain why events occur or predict what might happen in the future.

Predictive Analytics Are Based on Measured Data

Predictive Analytics Are Based on Measured Data

About 20 percent of business analytics are predictive; they try and forecast what may happen in the future (see above). How is this possible? Data scientists are not clairvoyant; they are just highly adept at building models from historical data to generate data about the future. As Wu explains, “It’s using data you have to predict data that you don’t have.”4

Marketers have been using a form of predictive analytics for years when we create time-series analysis to discern patterns in data and build trend lines based upon those findings. Today’s predictive models take more underlying data sources into account and are updated rapidly, improving their validity and impact.5 For example, using predictive analytics we can discern what products a consumer may purchase next or where a prospect is in the purchase process, when might they convert, and why. Predictive analytics can help us identify customers who are likely to defect and, of those, whom we might be able to persuade otherwise.

Prescriptive analytics takes future planning one step further, to identify an optimized course of action. It does so by generating multiple futures based upon different choices of action and evaluates the outcome. As Wu describes, “A prescriptive model can be viewed as a combination of multiple predictive models running in parallel, one for each possible input action. As such prescriptive models use existing models as well as action and feedback to guide the decision maker.”6

Where Does Big Data End and Small Data Begin?

A clear delineation between big data and small data does not exist. The difference is primarily a function of the amount of data under analysis, its form, and at times, its speed. As a rule of thumb, Thomas Davenport, professor of Management and Information Technology at Babson College and author of numerous books on the subject, describes small data as a terabyte (1012 bytes) of structured data or less. In contrast, big data is larger and more complex. As Davenport explained in Harvard Business Review, “If your organization stores multiple petabytes [1015 bytes] of data, if the information most critical to your business resides in forms other than rows and columns of numbers, or if answering your biggest question would involve a ‘mashup’ of several analytical reports, you’ve got a big data opportunity.”7

Say Good-bye to Traditional Marketing Research?

Will big data render traditional market research obsolete? After all, some of the data that has traditionally been collected through traditional marketing techniques can now be readily captured through big data. Rather than making survey research irrelevant, Bill Pink, senior partner, creative analytics at the global research agency Millward Brown, believes that big data will have the opposite impact.

“What matters is our ability to answer questions,” Pink explains. In his experience, big-data assets generate more questions than answers, and those questions need to be answered via traditional research forms. “Big data is the passive monitor and surveys become the focused, ongoing probes into changes and events that require exploration.”8

Consider customer-journey and touchpoint analysis, for example. Although big data illuminates the paths customers take and the devices they utilize, more traditional forms of research are still necessary to uncover the whys. Similarly, a considerable amount of noise exists in customer sentiment that is captured via social media. In Pink’s experience, “when market researchers apply their experience around understanding consumers and proven constructs of brand success such as levels of differentiation, dynamism, and salience to big-data assets, the results have clear meaning and often align with offline measures of equity and behaviors.”9

Rather than killing traditional marketing research, Pink believes that big data has liberated it. Because big data captures consumption data, follow-up surveys can be shorter and focus on key issues, enhancing their accuracy and appeal to customers. Insights derived from big data can be incorporated into survey design to drill down further and improve findings. This creates a blend of research curation and creation that better serves the brand and its customers. That is, of course, if the research is designed well and properly addresses the business issue at hand.

What is more, mobile is bringing new creativity to traditional survey design, creating opportunities to gather responses during experiences, rather than in hindsight, and to incorporate sound, photographs, and videos into feedback. Care must be taken to customize survey features to mobile—if surveys do not render properly people will not take them—and understand any new bias that mobile may add in terms of representation and fullness of attention.

Diane Hessan, CEO of Communispace, which creates and manages private online customer communities for leading brands, agrees that big data alone does not capture the whole picture. “With big data, companies can know everything their prospects and companies do, but that does not mean that they know them or know why they behave the way they do. Being able to talk with your customers on a daily basis is the difference between research and game-changing insights,” Hessan explains.

Next week: what are Marketers doing with Big Data?

This content is an example of what ContentOro does for its customers…providing high-quality, relevant content from experts and their published books.

About the Authors

Larry Weber and Lisa Leslie Henderson are the co-writers of this Digital Marketing guide. Larry is the CEO of Racepoint Global, an advanced marketing services firm. A globally known expert in public relations and marketing services, Larry has successfully built companies and brands and is passionate about the future of marketing. Lisa is an observer, synthesizer, and writer who draws extensively from her background in marketing and consulting. Lisa and Larry have collaborated on two guides to date, The Digital Marketer, and Everywhere: Comprehensive Strategy for the Social Media Era. To stay current on their thinking, frequent and follow them at @TheLarryWeber and @ljlhendo.

Buy on Amazon: The Digital Marketer: Ten New Skills You Must Learn to Stay Relevant and Customer-Centric

2. MG Siegler, “Eric Schmidt: Every 2 Days We Create as Much Information
as We Did Up to 2003,” TechCrunch (blog),
3. Michael Wu, “Big Data Reduction 1: Descriptive Analytics,” Lithosphere:
Science of Social Blog ,
4. Michael Wu, “Big Data Reduction 2: Understanding Predictive Analytics,”
Lithosphere: Science of Social Blog,
5. Michael Wu, “Big Data Reduction 3: From Descriptive to Prescriptive,”
Lithosphere: Science of Social Blog,
6. Personal e-mail from Dr. Wu to the writers
7. Thomas Davenport and D. J. Patel, “Data Scientist: The Sexiest Job of
the 21st Century,” Harvard Business Review,
8. Bill C. Pink, “How Big Data Liberates Research,” Millward Brown,
9. Author interview with Bill Pink

ContentOro revolutionizes the way digital marketers and brands acquire content for marketing. We partner with the world’s best publishers to bring the contents of their books to life on the web with our innovative technology. Creating compelling experiences that tie our content to our clients’ product is our mission.

Trackbacks & Pings

Leave a Reply Text

Your email address will not be published. Required fields are marked *