Analytics — The First Pass

Vendor AnalyticsLast year a group of executives at one of our big company clients decided to take a hard look at efficiency and outcome issues around learning.

Having to deal with a variety of use cases, end-user groups and training providers, not to mention the complications of operating in more than 50 countries and 18 languages, the executives saw their immediate task as getting on top of the data.

To this end, they put the following into effect:

  1. All courses, seminars and training events now end with a mandatory, standardized evaluation comprised of five questions, the first being the by-now-classic Net Promoter Score (“NPS” or the number of raving fans minus the number of complainers divided by the total number of responses multiplied by 100, this process yielding a number somewhere between minus and plus 100 – big positive numbers are good). The NPS question is followed by simple, sensible questions on each training program’s relevance by job role and topic, quality and effect.
  2. Individual employee progress is now measured the same way and on the same scale for all training.
  3. Costs are standardized on a per-employee basis and resolved to a base currency.
  4. Time costs are no longer only estimated by the provider, but also measured by the learning management system on a per-employee basis for training delivered online. Time taken is recorded for live and virtual events and estimated (or reported) for take-home assignments, again, all on a per-employee basis.
  5. All training is associated with a named provider, meaning a courseware-off-the-shelf, consulting or custom content vendor in the case of external training and an employee or department in the case of internally sourced training.
  6. All training is placed into one of five company-defined general topic categories.
  7. All training is sponsored by and associated with one of the company’s internal training academies (IT, product and services education, etc.).

The result after a year is that the company has a standardized report format for training, meaning company executives now have a shared language for cost vs. benefit issues, training take-up, attrition and mastery rates, lost-labor costs, unnecessary duplications, regulatory compliance vs. employee development spend, etc.

The company can also stack-rank vendors now, i.e., if the company is using three IT training vendors globally, company executives can see at a glance which vendor offers the most value for money, which vendor to drop and which vendor to squeeze on price come renewal.

All very clever but just the beginning.

The next steps will include getting a lot more sophisticated about data types and then marrying the learning data to line-of-business data.

This will be interesting.

Leave a Reply

Your email address will not be published. Required fields are marked *

*

*