Corporate Learning Strategies

Daniel R. Tobin, Ph.D.

 


The Fallacy of ROI Calculations

Copyright 1998 Daniel R. Tobin

Over the course of the past decade, I have received a number of calls from training directors in companies large and small, all with the same request for help: "My CEO (or CFO) has just told me that I have to produce a study to show the return on investment (ROI) for my training budget. Can you help?"

Having worked with a couple of these companies, and having talked at length with the other training directors who called me, I have drawn a singular conclusion: If you wait until the CEO asks for an ROI study to justify your training budget to try to demonstrate how your training group adds value to your company, it is too late - the CEO has already decided to greatly reduce your budget, or to eliminate the training group altogether, and is just using the ROI study to justify that decision.

Why I Don't Do ROI

My doctorate is in the field called "the economics of education." I spent a large part of my graduate education learning how to do ROI studies, and actually did some to earn my way through graduate school. So it isn't because I don't know how to do an ROI study that I refuse to do them. Another excuse that is often given for not doing ROI studies is that the benefits of training and education programs are often too intangible to measure in dollar terms - I don't use that excuse either.

The reason why I won't do an ROI study is that any major change effort within an organization requires not just training, but many other factors. There was a famous quotation from Motorola University in an article many years ago. The article told the story of a Motorola factory that was converting its production from old analog technology to new digital technology. The factory was closed down for some months, while new manufacturing equipment was installed, and, at the same time, Motorola University re-tooled the workforce. When the new factory was re-opened, new management techniques, such as high performance work teams (HPWT), were introduced along with the new technology. A study was done on the productivity of the workforce in this factory, before and after the changes and the training. Using the cost of the training and the increase in workforce productivity as the bases for the ROI calculation, they reported that there was a "30-times" return on investment, i.e., for every dollar spent on training, there was a $30 increase in productivity. (Wouldn't every training manager in the world love to have that number to report to the CEO?)

I have no doubt that Motorola University did a splendid job training those employees - Motorola University is the "granddaddy" of all corporate universities and has produced some outstanding programs and results over several decades of work. My argument with the study is that the increases in workforce productivity did not come from the training program alone - that the improved manufacturing equipment and the investments in instituting the HPWT environment were probably just as important factors in the productivity increase, but their costs never entered into the equation.

A Hypothetical Example
Let's look at a similar, but hypothetical example. The VP of manufacturing for a company has decided that he wants to introduce high performance work teams into the company's three manufacturing plants. The corporate manufacturing organization has arranged for training on HPWT, and he sends a memo to his three plant managers mandating attendance by their staffs at this training.

Plant Manager #1 reads the VP's memo and is very enthusiastic. "This is the wave of the future," he tells his staff. "I'm so psyched about this that I'm going to attend the training with you, and then we'll figure out how to make it really work here. This is going to be great!"

Plant Manager #2 isn't quite as enthusiastic, but is willing to give it a try. "Go to the training," she tells her staff, "and then we'll discuss it when you get back. If you think there is some value in this HPWT stuff, we'll try a pilot to evaluate it."

Plant Manager #3 hates the idea. "This is just another flavor of the week from corporate," he says. "Go to the training - we don't have a choice -- but don't expect anything to change once you get back."

The training is held with the mandated attendance from all three plants. Once back at the plants, Plant Manager #1 pushes aggressively ahead to institute HPWT, and the results are great. Plant Manager #2 sets up a pilot that seems to be working well, but is so small in scope that there are no real effects on the plant's overall productivity. Plant Manager #3 changes nothing.

The question now is: "How do you measure ROI on the investment in HPWT training in this company?" My answer is that you can't do it rationally.

A Correctional Example
Several years ago, I received a call from the training director of a state correctional system's training academy. "I saw an article you wrote about learning organizations," the director said. "Would you be willing to do a half-day seminar on learning organizations for the state corrections commissioner and his staff, and another similar seminar for the superintendents of the state's correctional facilities?"

I replied that I could certainly do that, but suggested that we sit down and talk for a while about her goals for the program. When we met, she told me why she wanted me to do these two seminars. She was the co-chair of a system-wide task force charged with building a new "inmate management system." The commissioner felt this was very important and had set aside millions of dollars for the project. But the task force was finding resistance and apathy toward the project throughout the correctional system, from members of the commissioner's staff to the superintendents. She had just finished reading Peter Senge's book, The Fifth Discipline, and had loved it. "If I could get all of the commissioner's staff and all the superintendents thinking like that, it would make things a lot easier for the task force," she told me.

"You're probably right," I replied. "But you can't teach Senge's methods in three or four hours and then expect immediate changes in behavior. It just doesn't happen. Look how long it took you just to read the book."

After more discussion and after interviewing the commissioner, several of his staff members, and a couple of the superintendents, I designed an intervention that included two half-day sessions (one for the commissioner and his staff and the other for the superintendents) that focused on leadership and change. In those sessions, we discussed the importance of the new inmate management system and their roles in making the new system happen. After winning the commitment of both groups, we then did a full-day action planning session with the two groups working together. The intervention was successful in getting the project back on track.
Now, how do we measure the ROI on this intervention? The customer felt that the intervention was very effective in getting the project moving - that it would never have come to reality without the intervention. But the cost of the intervention was miniscule in comparison to the millions that were being spent on the project itself. I would argue that there is no way to separate out the benefits of the intervention from the overall benefits of the new system, and it would be ludicrous to attribute all of the benefits of the new system to the intervention.

A Different Perspective on Training Costs and Benefits

When an automobile maker is designing a new model, he does not ask the people in charge of wheels and tires to do an ROI study to justify having wheels and tires on the new model. It is accepted that the new model will not be complete, will not work, without wheels and tires. Learning should be viewed as the wheels and tires of any organizational change effort - no change effort can be successful without learning.
If the automobile maker does not ask the wheels and tires department to justify its contribution to the new model on the basis of ROI, what does he ask of the wheels and tires department? Three things:


That the wheels and tires match the performance characteristics of the new model - you don't want to put highway tires on an SUV.
That the wheels and tires match the styling of the new model - you don't want to put fancy cast magnesium wheels on an economy family sedan.
That the wheels and tires match the price profile of the new model - you don't put an ultra-expensive wheel and tire combination on a Yugo.

Similarly, you want the investment in learning to match the overall change effort, to be appropriate to the level, speed, and intensity of the change effort. More importantly, you want to ensure that your planned learning programs will actually help the change effort succeed in meeting its goals.

When the automobile maker has completed plans for the new model, he does do an ROI study to justify the overall investment in the model. Similarly, in any corporate change effort, ROI calculations can be useful in evaluating the overall effort. In the Motorola example discussed earlier, the ROI model would include investments in plant and equipment as well as in the HPWT program, along with the training investment.

Tying Learning to Organizational Goals: The Learning Contract

If you want your training to group to be valued by organizational leaders, you must start and end all of your efforts by focusing on the organization's goals. I have devised an instrument that I call a "learning contract" to guide you through this process. The learning contract can be used at the level of the individual employee, but my focus here is on how to use it to guide the training group's efforts as a whole.
The learning contract has three main sections:

The Business Need
The Learning Plan
The Business Result

We'll examine each section separately. [For a more in-depth look at the learning contract, see my book: The Knowledge-Enabled Organization: Moving from Training to Learning to Meet Business Goals (AMACOM, 1997).]

Focus on the Business Need
Some years ago, I was contacted by a professional development group of a large IS consulting organization. To prepare for the consulting assignment, I asked the manager of this 40-person group to send me the company's last two annual reports, some of the marketing brochures the company uses with its clients, and a list of the current year's business goals for the business units that his group supported. His response: "We really don't have access to that kind of thing." My response to him: "If you don't understand the company's business goals and its strategic business initiatives, how can you possibly support them?"
To understand the business need, you need to ask three questions:
What are the goals of the organization?
What needs to change to meet those goals?
What needs to be learned to make those changes?
If you can answer those questions, you will ensure that your learning plans will focus directly on the needs of the business.

Focus on the Learning Plan
Once you understand the business need, you can develop a plan to meet the learning requirements associated with that need:
What will be learned?

How will it be learned?
When will it be learned?
How will the learning be measured?

The first three of these bullets are almost universally used by traditional training groups in making their plans - they relate to learning content, format, and schedules. Many training groups use some type of evaluation methods to measure learning achievement. But this is not enough - we need to evaluate learning not just by objective methods related to the mastery of the learning content, but also on the basis of business results.

Focus on the Business Result
This is where most traditional training groups fall down. For any learning program you design, you should answer these two questions as you plan the program:

How will learning be applied to the job?
What changes in business results can be expected to result from our learning efforts?

Many trainers feel that their responsibilities end when the student leaves the classroom or when a technology-based training solution is delivered to the student. The first bullet above, how learning will be applied to the job, says that the training group must work with organizational management to ensure that a plan is in place for the application of learning to the job and the reinforcement of that learning once the employee leaves the learning activity and starts to apply his or her learning to the job.

The second bullet is the payoff - the ultimate measure of the success of the training group. One of the basic tenets of program evaluation is that the goals of the program should directly connect to the evaluation measures for that program. So, if you start with the company's business goals, you should also end with the company's business goals and how the programs you design and deliver affect those business goals.

If Not ROI, Then What?

This article is based on a presentation I have made to conference audiences on three continents. At the conclusion of each presentation, I am typically asked: "If you don't use ROI to measure the value of your programs, then what do you use?" Throughout my 25+ years in the training field, I have always used the same indicator of my success - when company management/leadership gets up and says: "We had a really good quarter or year, meeting (or exceeding) our business goals - and, by the way, we couldn't have done it without the efforts of the training group" - then I know that the training group has been successful.

Tobin's Law
This entire article can be summarized by what I immodestly call "Tobin's Law" -- If you start and end all of your learning efforts by focusing on your organization's goals, you will never be asked to do an ROI analysis to justify your budget.

 

 


Dan Tobin welcomes your reactions to and comments on this article. To send him e-mail, click here or send email to DanielTobin@att.net.

 

Return to Publications Page

Home Page

About Dan Tobin