Analytical Thinking for L&D

Beyond programs and measures, big data is also prompting learning and development departments to develop deeper analytic skills. In the past, learning organizations mostly recruited instructional designers, classroom trainers and subject matter experts.

“Now I need people that understand analytics, data mining and the business itself so they know what context to put the data into,” Losey said. “They also need to know where the data came from and what it means, because a lot of data is worthless.”

Thinking that big data analytics is just traditional HR analytics on a larger scale is a fatal error. In traditional HR analytics, organizations have a question they want to answer such as how many people are planning to retire in the next five years or the number of women among the top 100 leaders.

Big data analysis takes disparate data from diverse sources in different formats and uses statistical techniques such as regression analysis and predictive modeling to allow the data to speak for itself, Howe said, and give rise to interesting correlations and outliers.

That sort of sophisticated thinking about data has left many in learning and development behind, said Josh Bersin, founder of analyst firm Bersin by Deloitte. “The world has come and gone and left them in the dust,” he said. “They spent a lot of time focused on trying to measure this one thing that they do which is training … but without measuring anything else it doesn’t have a lot of meaning.”

Bersin recommended creating a talent analytics center of excellence that collects data from the various functional silos, including learning, and use it to examine business problems such as sales productivity, turnover or compliance. That kind of correlated talent management data has the potential to provide real breakthroughs, Masie said.

“If I actually look at the performance changes of an individual — whether it’s cars rented, complaints handled, patients kept alive, whatever you want to use — and you correlate that data back to learning, it gets really interesting,” he said.

Big Data Just Big Hype?
Within L&D, big data is approaching the pinnacle of the hype curve. Like social learning before it, the topic is the theme of industry conferences and subject of countless blogs and articles like this one. Vendors and software providers have seized upon it to tout the capabilities of their applications and services.

It’s easy to listen to the buzz, engage in magical thinking about the potential of big data and say, “Let’s collect a whole bunch of data and see what it says,” Bersin said. “Maybe something beautiful will happen.”

Collecting and analyzing data of this magnitude is enormously difficult, he warned, and requires not only a way to bring together data from multiple systems but also to clean up “dirty” HR data that is old, inconsistent and full of errors. It also requires thoughtful preparation and asking the right kinds of questions.

“Data lies when you ask it the wrong questions,” said Mike Loukides, vice president of content strategy at O’Reilly Media, a technology publisher and media company, and someone who has written on the topic. “If you are sloppy with the questions that you are asking when you start your data analysis, then you’ll get answers that may well look good but don’t really mean anything.”

Bersin recommended that companies start with a distinct problem, such as identifying the characteristics of a successful hire or the factors that lead to employee turnover. “Eventually you’re going to build a bigger and bigger database but you’re going to be much more effective if you start with a small number of problems and try to get the data that will help you solve that problem,” he said.

Above all, it’s important to understand that data — big or little — is only one factor to be considered when determining a course of action. Data science is not about “turning a crank and getting out a big hammer that you can bash your opponents over the head with,” Loukides said. “It’s not about putting an end to the conversation. Data is the basis for the conversation. It’s not about saying we did our numbers and this is the answer.”

Big data can’t remove the ambiguity that is a reality of business. What it can do is shed light on alternative perspectives and uncover possibilities that hedge rather than entirely remove uncertainty.

“You are able to reduce it by doing certain things, by having good-quality data and using appropriate methods to analyze that data,” said EMC’s Dietrich. “Insofar as you can do that, you can make better decisions with less risk.”

In addition to better decision-making, big data can turn up correlations that demonstrate learning’s relevance in potentially new and meaningful ways. At EMC, Clancy, whose organization delivers training to customers as well as employees, wanted to know the effect a customer investment in education had on subsequent EMC business. “A newbie data scientist came back after four months and said I have the answer for you,” he said. “For every dollar that a customer buys from us, they will buy an incremental $10 of EMC product.”

Further analysis showed that EMC’s Net Promoter Score, a measure of customer loyalty, was three times higher for customers who used his group’s training versus those who didn’t. That sort of big data science addresses the age-old credibility problem, giving talent managers more objective information to make more effective recommendations to business leaders.

“Executive-level data science allows you to have executive-level conversations for strategic direction, funding and everything else that goes along with it,” Clancy said.

The Three V’s and L&D