Corporate Learning
Strategies
Daniel R. Tobin,
Ph.D.
The
Fallacy of ROI Calculations
Copyright ©1998 Daniel R. Tobin
Over
the course of the past decade, I have received a number of calls from training
directors in companies large and small, all with the same request for help:
"My CEO (or CFO) has just told me that I have to produce a study to show
the return on investment (ROI) for my training budget. Can you help?"
Having
worked with a couple of these companies, and having talked at length with the
other training directors who called me, I have drawn a singular conclusion: If
you wait until the CEO asks for an ROI study to justify your training budget to
try to demonstrate how your training group adds value to your company, it is too
late - the CEO has already decided to greatly reduce your budget, or to
eliminate the training group altogether, and is just using the ROI study to
justify that decision.
Why I Don't Do ROI
My
doctorate is in the field called "the economics of education." I
spent a large part of my graduate education learning how to do ROI studies, and
actually did some to earn my way through graduate school. So it isn't because I
don't know how to do an ROI study that I refuse to do them. Another excuse that
is often given for not doing ROI studies is that the benefits of training and
education programs are often too intangible to measure in dollar terms - I
don't use that excuse either.
The
reason why I won't do an ROI study is that any major change effort within an
organization requires not just training, but many other factors. There was a
famous quotation from
I
have no doubt that Motorola University did a splendid job training those
employees - Motorola University is the "granddaddy" of all corporate
universities and has produced some outstanding programs and results over
several decades of work. My argument with the study is that the increases in
workforce productivity did not come from the training program alone - that the
improved manufacturing equipment and the investments in instituting the HPWT
environment were probably just as important factors in the productivity
increase, but their costs never entered into the equation.
A Hypothetical Example
Let's look at a similar, but hypothetical example. The
VP of manufacturing for a company has decided that he wants to introduce high
performance work teams into the company's three manufacturing plants. The
corporate manufacturing organization has arranged for training on HPWT, and he
sends a memo to his three plant managers mandating attendance by their staffs
at this training.
Plant
Manager #1 reads the VP's memo and is very enthusiastic. "This is the wave
of the future," he tells his staff. "I'm so psyched about this that
I'm going to attend the training with you, and then we'll figure out how to
make it really work here. This is going to be great!"
Plant
Manager #2 isn't quite as enthusiastic, but is willing to give it a try.
"Go to the training," she tells her staff, "and then we'll
discuss it when you get back. If you think there is some value in this HPWT
stuff, we'll try a pilot to evaluate it."
Plant
Manager #3 hates the idea. "This is just another flavor of the week from
corporate," he says. "Go to the training - we don't have a choice --
but don't expect anything to change once you get back."
The
training is held with the mandated attendance from all three plants. Once back
at the plants, Plant Manager #1 pushes aggressively ahead to institute HPWT,
and the results are great. Plant Manager #2 sets up a pilot that seems to be
working well, but is so small in scope that there are no real effects on the
plant's overall productivity. Plant Manager #3 changes nothing.
The
question now is: "How do you measure ROI on the investment in HPWT
training in this company?" My answer is that you can't do it rationally.
A Correctional Example
Several years ago, I received a call from the training director of a state
correctional system's training academy. "I saw an article you wrote about
learning organizations," the director said. "Would you be willing to
do a half-day seminar on learning organizations for the state corrections
commissioner and his staff, and another similar seminar for the superintendents
of the state's correctional facilities?"
I
replied that I could certainly do that, but suggested that we sit down and talk
for a while about her goals for the program. When we met, she told me why she
wanted me to do these two seminars. She was the co-chair of a system-wide task
force charged with building a new "inmate management system." The
commissioner felt this was very important and had set aside millions of dollars
for the project. But the task force was finding resistance and apathy toward
the project throughout the correctional system, from members of the
commissioner's staff to the superintendents. She had just finished reading
Peter Senge's book, The Fifth Discipline, and had loved it. "If I could
get all of the commissioner's staff and all the superintendents thinking like
that, it would make things a lot easier for the task force," she told me.
"You're
probably right," I replied. "But you can't teach Senge's methods in
three or four hours and then expect immediate changes in behavior. It just
doesn't happen. Look how long it took you just to read the book."
After
more discussion and after interviewing the commissioner, several of his staff
members, and a couple of the superintendents, I designed an intervention that
included two half-day sessions (one for the commissioner and his staff and the
other for the superintendents) that focused on leadership and change. In those
sessions, we discussed the importance of the new inmate management system and
their roles in making the new system happen. After winning the commitment of
both groups, we then did a full-day action planning session with the two groups
working together. The intervention was successful in getting the project back
on track.
Now, how do we measure the ROI on this intervention? The customer felt that the
intervention was very effective in getting the project moving - that it would
never have come to reality without the intervention. But the cost of the
intervention was miniscule in comparison to the millions that were being spent
on the project itself. I would argue that there is no way to separate out the
benefits of the intervention from the overall benefits of the new system, and
it would be ludicrous to attribute all of the benefits of the new system to the
intervention.
A Different Perspective on Training Costs and Benefits
When
an automobile maker is designing a new model, he does not ask the people in
charge of wheels and tires to do an ROI study to justify having wheels and
tires on the new model. It is accepted that the new model will not be complete,
will not work, without wheels and tires. Learning should be viewed as the
wheels and tires of any organizational change effort - no change effort can
be successful without learning.
If the automobile maker does not ask the wheels and tires department to justify
its contribution to the new model on the basis of ROI, what does he ask of the wheels
and tires department? Three things:
· That the wheels and tires match the performance characteristics of the
new model - you don't want to put highway tires on an SUV.
· That the wheels and tires match the styling of the new model - you
don't want to put fancy cast magnesium wheels on an economy family sedan.
· That the wheels and tires match the price profile of the new model -
you don't put an ultra-expensive wheel and tire combination on a Yugo.
Similarly,
you want the investment in learning to match the overall change effort, to be
appropriate to the level, speed, and intensity of the change effort. More
importantly, you want to ensure that your planned learning programs will
actually help the change effort succeed in meeting its goals.
When
the automobile maker has completed plans for the new model, he does do an ROI
study to justify the overall investment in the model. Similarly, in any
corporate change effort, ROI calculations can be useful in evaluating the
overall effort. In the Motorola example discussed earlier, the ROI model would
include investments in plant and equipment as well as in the HPWT program,
along with the training investment.
Tying Learning to Organizational Goals: The Learning Contract
If
you want your training to group to be valued by organizational leaders, you
must start and end all of your efforts by focusing on the organization's goals.
I have devised an instrument that I call a "learning contract" to
guide you through this process. The learning contract can be used at the level
of the individual employee, but my focus here is on how to use it to guide the
training group's efforts as a whole.
The learning contract has three main sections:
·
The Business Need
· The Learning Plan
· The Business Result
We'll
examine each section separately. [For a more in-depth look at the learning
contract, see my book: The Knowledge-Enabled Organization: Moving from
Training to Learning to Meet Business Goals (AMACOM, 1997).]
Focus on the Business Need
Some years ago, I was contacted by a professional development group of a large
IS consulting organization. To prepare for the consulting assignment, I asked
the manager of this 40-person group to send me the company's last two annual
reports, some of the marketing brochures the company uses with its clients, and
a list of the current year's business goals for the business units that his
group supported. His response: "We really don't have access to that kind
of thing." My response to him: "If you don't understand the company's
business goals and its strategic business initiatives, how can you possibly
support them?"
To understand the business need, you need to ask three questions:
· What are the goals of the organization?
· What needs to change to meet those goals?
· What needs to be learned to make those changes?
If you can answer those questions, you will ensure that your learning plans
will focus directly on the needs of the business.
Focus on the Learning Plan
Once you understand the business need, you can develop a plan to meet the
learning requirements associated with that need:
· What will be learned?
·
How will it be learned?
· When will it be learned?
· How will the learning be measured?
The
first three of these bullets are almost universally used by traditional
training groups in making their plans - they relate to learning content,
format, and schedules. Many training groups use some type of evaluation methods
to measure learning achievement. But this is not enough - we need to evaluate
learning not just by objective methods related to the mastery of the learning
content, but also on the basis of business results.
Focus on the Business Result
This is where most traditional training groups fall
down. For any learning program you design, you should answer these two
questions as you plan the program:
·
How will learning be applied to the job?
· What changes in business results can be expected to result from our learning
efforts?
Many
trainers feel that their responsibilities end when the student leaves the
classroom or when a technology-based training solution is delivered to the
student. The first bullet above, how learning will be applied to the job, says
that the training group must work with organizational management to ensure that
a plan is in place for the application of learning to the job and the
reinforcement of that learning once the employee leaves the learning activity
and starts to apply his or her learning to the job.
The
second bullet is the payoff - the ultimate measure of the success of the
training group. One of the basic tenets of program evaluation is that the goals
of the program should directly connect to the evaluation measures for that
program. So, if you start with the company's business goals, you should also
end with the company's business goals and how the programs you design and
deliver affect those business goals.
If Not ROI, Then What?
This
article is based on a presentation I have made to conference audiences on three
continents. At the conclusion of each presentation, I am typically asked:
"If you don't use ROI to measure the value of your programs, then what do
you use?" Throughout my 25+ years in the training field, I have always
used the same indicator of my success - when company management/leadership gets
up and says: "We had a really good quarter or year, meeting (or exceeding)
our business goals - and, by the way, we couldn't have done it without the
efforts of the training group" - then I know that the training group has
been successful.
Tobin's Law
This entire article can be summarized by what I immodestly call "Tobin's
Law" -- If you start and end all of your learning efforts by focusing
on your organization's goals, you will never be asked to do an ROI analysis to
justify your budget.
Dan Tobin
welcomes your reactions to and comments on this article. To send him e-mail, click
here or send email to DanielTobin@att.net.