CH-7 Copyright 2008 Werner, et al 1 Evaluating HRD Programs Chapter 7 CH-7 Copyright 2008 Werner, et al 2 Learning Objectives – 1 • Define evaluation and explain its role in HRD • Compare different frameworks for HRD evaluation • Discuss the various types of evaluation information available and compare the methods of data collection • Explain the role of research design in HRD evaluation CH-7 Copyright 2008 Werner, et al 3 Learning Objectives – 2 • Describe the ethical issues involved in conducting HRD evaluation • Identify and explain the choices available for translating evaluation results into dollar terms • Calculate a utility estimate for a target organization. Discuss how technology impacts HRD evaluation CH-7 Copyright 2008 Werner, et al 4 Questions to Consider • How do you evaluate training and HRD? • What measures can be used to evaluate training? • Is there one best way to evaluate training? • What should be considered as one prepares to evaluate HRD? • What are the ethical issues involved in evaluating HRD? • How can the value of HRD be expressed in terms of costs and benefits, or dollars and cents? CH-7 Copyright 2008 Werner, et al 5 HRD Program Effectiveness • What is meant by effectiveness? – Is it the same thing as efficiency? • How is effectiveness measured? • What is the purpose of determining effectiveness? – That is, what decisions are made after a program is judged effective or ineffective? CH-7 Copyright 2008 Werner, et al 6 Effectiveness • A relative term – Effectiveness is determined with respect to the achievement of a goal or a set of goals – Must be determined with respect to the goals of the program or programs being examined CH-7 Copyright 2008 Werner, et al 7 A Quandary • Program can be effective in meeting some goals – staying within budget – increasing a participant’s skills • and be ineffective in meeting others – Improving customer satisfaction • How do you ensure effectiveness? CH-7 Copyright 2008 Werner, et al 8 Training and HRD Process Fig. 7-1 CH-7 Copyright 2008 Werner, et al 9 Purpose of Evaluation • HRD evaluation: – “The systematic collection of descriptive and judgmental information necessary to make effective training decisions related to the selection, adoption, value, and modification of various instructional activities.” CH-7 Copyright 2008 Werner, et al 10 Significant Points in Definition – 1 • Both descriptive and judgmental information may be collected – Descriptive information provides a picture of what is happening or has happened – Judgmental information communicates some opinion or belief about what has happened CH-7 Copyright 2008 Werner, et al 11 Significant Points in Definition – 2 • Evaluation involves the systematic collection of information – According to a predetermined plan to ensure that the information is appropriate and useful • Evaluation is conducted to help make informed decisions about particular programs and methods CH-7 Copyright 2008 Werner, et al 12 Evaluation Can Help • Determine whether a program is accomplishing its objectives • Identify the strengths and weaknesses of HRD programs • Determine the cost-benefit ratio of an HRD program • Decide who should participate in future HRD programs • Identify which participants benefited the most or least from the program • Gather data to assist in marketing future programs • Establish a database to assist management in making decisions CH-7 Copyright 2008 Werner, et al 13 Other Major Factors – 1 • If HRD staff cannot substantiate its contribution to the organization, its funding and programs may be cut during the budgeting process, especially when the organization faces tough times CH-7 Copyright 2008 Werner, et al 14 Other Major Factors – 2 • Evaluation can build credibility with top managers and others in the organization • Senior management often wants to know the benefits of HRD programs • Building credibility is a key aspect of evaluation - CH-7 Copyright 2008 Werner, et al 15 How Often Are HRD Programs Evaluated? • Most company-sponsored training • Less than half of executive MBA programs • Most used measures participant reaction – Not always useful CH-7 Copyright 2008 Werner, et al 16 Why Not Done Frequently? • Conducting an evaluation is not easy • Many external factors can affect whether employee performance improves – makes it difficult to evaluate the impact of just the training • HRD managers afraid of criticism and program cuts CH-7 Copyright 2008 Werner, et al 17 Evaluation Prior to Purchase • Many HRD and training programs are purchased by organizations from third parties – They wouldn’t buy a program they didn’t think was going to work – they have evaluated the program before buying it • Equally important to evaluate after use CH-7 Copyright 2008 Werner, et al 18 Changing Evaluation Emphasis • Stage One focuses on anecdotal reactions • Stage Two involves experimental methodology • Stage Three creatively matches research methodology to organizational constraints • Stage Four shifts the focus of evaluation from post-program results to the entire HRD process CH-7 Copyright 2008 Werner, et al 19 Evaluation Frameworks – 1 Table 7-1 Four levels: •Reaction •Learning •Job Behavior •Results Kirkpatrick (1967, 1987, 1994) Training Evaluation Criteria Model CH-7 Copyright 2008 Werner, et al 20 Evaluation Frameworks – 2 Table 7-1 Four levels: •Context •Input •Process •Product CIPP (Galvin, 1983) Training Evaluation Criteria Model