CEIT 317 - Instructional Technology and Material Development
Upon completion of this component, the learner will be able to:
THE ROLE OF EVALUATION IN THE INSTRUCTIONAL DESIGN PROCESS
In the presentation on Development and Usability testing, you learned something about how to evaluate your materials so that you can make informed design decisions regarding what needs to be changed, what needs to be added, and what needs to be deleted from your instructional materials in order to best facilitate learning. This type of evaluation is generally known as "formative" evaluation, because you complete it during the formative stages of the instructional design process, and it helps you to form the content and processes of your instruction.
Another kind of evaluation that is important in the instructional design process is generally called "summative evaluation," and is conducted after an educational experience has been implemented, in order to determine whether the students achieved the objectives of instruction, whether their learning transferred to "the real world" and in the case of corporate training, what impact this transfer of training had on the bottom line and the goals of the organization.
You will notice that, in the cases of both formative and summative evaluation, the emphasis is on evaluating the quality of instruction, not the quality of the learner. A general assumption in the field of IST is that most learners are capable of learning (unless the learner faces special challenges or disabilities) and the onus is on the instructional designer to design high quality instruction that will facilitate the learning of all learners. In other words, evaluation usually says a lot more about the quality of instruction than it does the intelligence of the learner.
Another general assumption in the field of IST is that analysis and evaluation are inextricably intertwined, and that one can not do a very good summative evaluation at the end of the ADDIE process unless a thorough analysis was completed at the beginning of the ADDIE process. This assumption is based primarily on the fact that the objecitves for the instruction are identified during the analysis phase, and without clear and appropriate objectives, it is impossible for the instructional designer to determine what should be measured during the evaluation phase.
REASONS FOR CONDUCTING SUMMATIVE EVALUATION
There are three good reasons for conducting summative evaluations:
FOUR TYPES (LEVELS) OF EVALUATION
Donald Kirkpatrick has identified four types/levels of evaluation that may be addressed when evaluating instructional programs, including:
USING KIRKPATRICK'S LEVELS OF EVALUATION
Kirkpatrick's model lists four levels that provide a *sequence* for evaluating instructional experiences. Kirkpatrick's model is sequential because it would be impossible to complete a level 4 evaluation without having completed levels 2 or 3. Each level provides important information in its own right, each level is a stepping stone to the next level, so none of the levels should be bypassed in order to get to a level that is considered more important. Each level provides progressively more valuable information regarding the impact of the learning experience, but at the same time, as one moves from level 1 up to level 4, the process of evaluation becomes more difficult, more nebulous, more costly and more time-consuming - which is why relatively few organizations actually conduct level 4 evaluations. Yet, level 4 evaluation is the most worthwhile and meaningful type of evaluation, if one has the resources to complete evaluation at that level.
Leshin, Pollock & Reigeluth, 1992: Instructional Design Strategies and Tactics. Unit 5 - Evaluate the Instruction.
Review and Discussion Questions