Revamping Talent Metrics for Better

Josh Craver’s early tenure in 2013 as a training and development leader with Western Union Co. came as the financial services firm embarked on a major transformation in light of its broadening global business.

But as Craver settled in to initiate the employee learning needed to get the transformation off the ground, he stumbled upon a quandary familiar to many talent executives.

“I became aware of the fact that we didn’t have the data we needed to get credibility and alignment with the business,” said Craver, Western Union’s vice president of talent management.

Most concerning was Englewood, Colorado-based Western Union’s lack of talent metrics. Without the ability to link talent management’s value to the business, Craver would be unable to get the resources needed to drive the learning for the company’s more than 11,000 employees to support the transformation.

Developing a System

To convince executives to invest the time and money in talent management, Craver needed a system that would help validate the effect of strategic human resources through metrics business leaders could understand.

“Executives define the value of an intervention by how well it correlates to performance,” Craver said, “and I knew we could create a huge amount of buy-in if we could quantify the impact of training to the business.”

– Josh Craver, vice president of talent management, Western Union Co. 

His solution: the talent development reporting principles, or TDRp, a reporting system that aims to replicate the financial industry’s generally accepted accounting principles.

The product of nonprofit the Center for Talent Reporting — whose executive director, David Vance, is the former chief economist and head of business intelligence at Caterpillar Inc. and former president of the company’s learning university — TDRp intends to build a talent reporting framework capable of showing cost-conscious executives the return on investment of talent management.

Craver said overall, Western Union’s use of TDRp’s talent reporting statements has allowed it to identify the effectiveness of training programs that have yielded millions in additional revenue. It has also changed the way the company’s HR team communicates with senior management.

“It allowed me to run talent management like any other business line with a standard P&L [profit and loss],” Craver said.

Western Union has come a long way since it started as a telegram service in 1851. More than a century and many business transformations later, Western Union is now known for its global financial services that include money transfers, business payments and prepaid cards.

In that time, Western Union has also expanded its business to more than 520,000 locations in 58 countries, adopted new electronic and mobile capabilities, acquired new businesses to target more small and midsize organizations, and expanded its line of prepaid cards.

The need to transform has hardly slowed down. Faced with ever-more ways for customers to manage money, Western Union’s senior leaders in recent years have focused on transforming its service to be more consumer-oriented, thanks to a growing number of competitors in the industry.

To support this business transformation, the company’s leadership team initiated a companywide culture change that included identifying key behaviors to drive a more customer-centric business model. The new approach called for training employees from the top down. Craver came on in 2013 to lead the training and development efforts to support this culture change.

The foundation of the new approach would be TDRp, a program to bring standard principles and reporting to human capital processes. Vance launched the organization in 2012 to help bring standards to the measuring and reporting talent, with the aim of helping companies align development efforts with business goals and establishing reporting tools that enable development teams to report the impact of learning to senior executives.

Craver first learned about TDRp at a Skillsoft conference in 2013. He said he was impressed by how the methodology aligned to traditional business reporting. “I also appreciated how the process created a partnership with business leaders to determine the business impact of the talent initiatives,” Craver said. “It is like having a P&L for talent development.”

A productive partnership among Jennifer Ramirez, Josh Craver and Scot Duffield's teams has helped employees take personal ownership of the customers' experience.

Craver presented the TDRp concept to Western Union’s head of human resources. He argued that it would be a good way to get sponsorship and to build credibility as a growth enabler for the business. They agreed to move forward with its implementation.

Talent Metrics Aligned

As part of the TDRp roll out, Craver’s team wanted to align all of Western Union’s HR technology systems.

Because of a number of acquisitions, Western Union had a multitude of different human resource information systems, making it difficult to get basic companywide talent data. So the firm consolidated most of the HRIS into Workday while moving all online learning to Cornerstone OnDemand, an integrated learning management system. Craver said it took just three months to integrate the two LMS systems into Cornerstone, in turn creating hundreds of thousands of dollars in efficiencies from time savings and outside vendor costs.

The official rollout of TDRp began in October 2013. The implementation phase included establishing processes for setting performance metrics on all learning and development activities, creating to collect and analyze the measurement data, and building reports to communicate progress to the various stakeholders.

The biggest early challenge was changing the culture of the training and development department to be more data-driven, Craver said. To do this, Craver brought in Vance to lead a series of half-day workshops on the use of project and summary reports as well as how to set and measure metrics that align with business goals.

“TDRp is designed to help bring business discipline to learning and development,” Vance said. “Trainees don’t report to trainers; they report to business leaders. So you have to be able to talk to leaders in business terms to get their support.”

The workshops helped Craver’s team learn how to begin every training conversation with whether training can help achieve a business goal, along with what metrics will be used to measure the impact. “The training was such an important part of this process,” Craver said. “Everything we do now begins with a business case and how we will measure our work.”

As part of the rollout, Western Union’s training and development team defined specific areas to measure based on four goals: revenue growth, leadership development, evolving the organization and achieving greater efficiencies. “Now when we structure any training, we align it back to one of these four pillars and are able to report back to the business leaders with data on our impact,” said Vitalija Jakovoniene, the company’s vice president of HR in Dubai, United Arab Emirates.

To ensure consistent quantifiable reporting, the team based its measures on the Donald Kirkpatrick levels of learning evaluation. Level one measures how employees feel about the training; level two measures whether employees learned new skills or knowledge; level three measures whether behavior changed as a result of the training; and level four measures whether that change in behavior lead to business results.

Western Union’s learning and development started off and continues to target level three results, which are determined by interviews with managers about whether employees applied what they learned on the job, according to Scot Duffield, head of talent operations in the firm’s Denver office.

“It provides us with good information to judge whether the training was a good investment,” Duffield said.

Reporting Update

For each talent management program, metrics data are collected and funneled into summary and operations reports for various stakeholders, and TDRp has a series of reports that deliver specific types of information.

For example, program reports deliver information on courses in support of specific business goals while summary reports are more high-level reports on effectiveness and efficiency measures that show how progress to date is aligned with corporate scorecard objectives. “The different kinds of reports allow us to manage our function like a business while also presenting the executive summary information to our sponsors in a way that’s easy for them to understand,” Craver said.

Every month, program owners update their respective program reports. The talent operations team checks the data and compiles it to populate the summary reports. Then, the talent management team reviews the reports, and regional consultants present the relevant data to the program sponsors and HR heads in each region.

Western Union’s training and development team reviews the results on a monthly basis to see how programs are working and what improvements can be made, Duffield said. The learning and development leaders in each region then take those reports back to their own stakeholders to share progress and priorities for their local teams.

“By communicating these reports to stakeholders, we have created a level of transparency around what we are doing,” Duffield said.

So far, the team says it hasn’t canceled any programs as a result of the reported metrics, but it is using the results to improve them.

Western Union, for instance, recently launched a three-tiered training program for field supervisors and midlevel managers on how to develop and lead a team. Early metrics show that almost half of all eligible managers signed up for the first tier of the program, but only one-third of those managers completed all three tiers.

“We will use the data we received to make changes to the program when we roll it out again,” Duffield said.

The team is also developing more advanced tools to gather level four results. These will demonstrate more direct influence on business performance. Duffield expects to implement a five-point Likert Scale — in which things are measured on a numeric scale; in this case from one to five — for these measures. It will ask employees and managers to rate things like whether trainees are applying recently learned knowledge on the job.

Customer Comes First

One of the most prominent training programs rolled out since Western Union implemented TDRp is a series of modules in support of the mission for employees to be more customer-centric.

Jennifer Ramirez, the firm’s vice president of product development, is head of the customer experience strategy. She worked closely with Craver’s team and the finance and strategy group to identify the best ways to promote the culture change and measure its impact before the idea of training ever came up.

Using analysis of customer retention data, customer surveys and industry benchmarks, Ramirez’s team determined that services representatives should focus on improving the customer service experience by identifying the customer journey with Western Union and where they might help them address pain points and bolster the company brand.

With that goal, Ramirez worked with Craver to build four training modules about the changing corporate mindset, developing a common customer-centric language and understanding and reacting to the customer experience. “These are business-driven training programs, not HR driven,” Craver said.

Craver’s team began by developing a business case that included training costs and metrics for how they would measure success. Instead of using external trainers for the two-hour courses, the team employed internal leaders as teachers.

Leaders at Western Union were asked to deliver at least six workshops for a total of 12 hours per year. “It saved us hundreds of thousands of dollars in trainer costs,” Craver said. It also had the added benefit of demonstrating the importance of the training to employees who saw leaders taking time away from their jobs to lead the workshops.

Initial metrics for success are aligned with evolving the organization; they also focus on employee surveys and employee net promoter scores to quantify improved engagement as a result of the training. Duffield said there is a specific question in the engagement survey on whether the Western Union organization is meeting the needs of customers. “Since we launched the customer centricity training program last year, we’ve seen the level of improvement in this score double,” Duffield said.

Scot Duffield, left, and Josh Craver stand in the WU Customer Experience Center, which gives all employees the opportunity to experience money transfers from both the agent and customer perspective.

Ramirez said she is also planning to use customer surveys to quantify improvements in customer service delivered and to compare customer retention before and after the training, both of which can be considered level four measures of business impact. “It’s still at the beginning stages, but our goal is to make sure we get measures at every level and that it ultimately changes behavior,” Ramirez said.

Being able to share these kinds of measures with stakeholders has changed the way the talent management team communicates with managers. Now when managers say they want training, the first question the talent management leader asks is, “What is the impact you want to achieve?” and “How will we measure those results?”

“It has completely changed the dynamic of the conversations we have with business-unit leaders,” Jakovoniene said. “Now it is much less about what HR wants and more about what the business wants.”

Results Are In

The results haven’t just come from the customer-centric training program. A global sales training program for core salespeople and account managers has led to substantial measurable bottom-line improvements.

In the past the learning and development team would only measure how many people took the course, Jakovoniene said. Now they gather feedback from trainees and managers about whether the training lead to a change in behavior, whether sales results have improved and who made the most progress.

According to reports from Craver, the sales force effectiveness training has led to millions in increased revenue specifically attributed to skills learned and applied. “The training program delivered ROI more than 20 times our investment,” Craver said.

In another instance, the global team rolled out a self-paced management fundamentals course in late 2014. “The executive team believes that leadership skills will make or break our cultural transformation, and that we need leaders at every level to have the skills and philosophy to drive this culture change message,” Jakovoniene said.

The initial measure of success was whether leaders would use the training. The results were clear. Within six months, 700 leaders elected to take the course.

“That’s an amazing result for a voluntary program, and a great metric to report back to the executives to demonstrate the need for training,” Jakovoniene said. The team also plans to review the results of the twice-annual employee surveys, which ask about the effectiveness of management, to demonstrate the positive influence of leadership development to the business.

“We expect that score will improve as a result of this course,” Jakovoniene said.

Data Into Focus

Less than two years after implementing TDRp, Craver said his team has already driven an impressive level of change. “We have demonstrated support from the top, and are building a learning culture around the WU world,” Craver said.

Now that the team feels confident about the value of metrics and how to identify and collect them, Craver said the next goal is to streamline the process. The team is now working to scale back the number of measures related to each program to three to five from more than 10.

The team is also working to streamline the data collection and evaluation process, which can take hours to complete because of the use of manual data recording in spreadsheets, Duffield said. At deadline, the team was reviewing technology tools to automate many of these steps to make it more efficient.

Finally, the team is ramping up marketing of the program and its early success stories to raise awareness among Western Union leadership and employees. “We want them to see the connection between learning and improved performance,” Craver said.

To generate awareness, the team is sharing links to course results and encouraging leaders to share stories in meetings. It also created a microlearning series, “Take 10,” mini online courses that can be completed in less than 10 minutes.

“Our data tells us what is effective and what isn’t,” Craver said. “People want learning in smaller, easily digestible chunks.”

While the learning and development team hasn’t figured out exactly how to measure the business impact of every course and development opportunity it offers, it feels the fact that it’s quantifying the value of training and talking to managers in those terms has reinvented its value proposition.

 “It has helped us develop more consistent training programs on a global basis,” Duffield said, “and it has helped us prove our value to the business.”