The Evaluation of Criteria & Phases of Evaluation

THE EVALUATION
OF CRITERIA &
PHASES OF
EVALUATION
Jose Paul Joseph

THE EVALUATION OF CRITERIA
 To evaluate its training program , a company
must decide how it will determine the program
effectiveness ; that is it must identify what
outcomes or criteria it will measure.
 D.L Kirkpatrick's four-level framework for
categorizing training outcomes
Level Criteria Focus
4 results Business results achieved by trainees
3 behavior Improvement of behavior on the job
2 learning Acquisition of knowledge, skills,

attitude , behavior
1 reactions Trainee satisfaction

CONTD..
 Kirkpatrick's frame work suggests that higher
level outcomes should not be measured unless
positive changes occurs in lower level
outcomes.
 Also framework implies that changes at a
higher level (e.g.: results) are more beneficial
than changes at a lower level (e.g.: learning ).
 Framework has been criticized for number of
reasons first research has not found that each
level is caused by the level that precedes it in
the framework.
CONTD..
 Second the approach does not take into account the
purpose of evaluation.
 Third, use of the approach suggests that outcomes can
& should be collected in an orderly manner,
realistically learning measures need to be collected at
approximately the same time as reaction measures ,
near the end of program.
 As a result of these criticisms, both training
practitioners & academic research have argued that
more comprehensive models are needed.
 Accordingly training outcomes have been classified into
five categories : cognitive outcomes, skill-based
outcomes, affective outcomes, result & return on
investment.
COGNITIVE OUTCOMES
 Cognitive outcomes are used to determine
the degree to which trainees are familiar
with principles, facts, techniques,
procedures or processes emphasized in the
training program.
 It measures the knowledge trainees learned
in the program.
 Cognitive outcomes are level 3 criteria in
Kirkpatrick's framework.
 Typically ,pencil-paper tests are used to
assess cognitive outcomes.
SKILL BASED OUTCOMES
 Skill based outcomes are used to assess the
level of technical or motor skills & behaviors.
 Skill based outcomes include acquisition or
learning of skills and use of skills on the job
(skill transfer).
 Skill based outcomes relate to Kirkpatrick's
level 2(learning) and level 3(behavior).
 The extend to which trainees have learned
skills can be evaluated by observing their
performance in work samples, skill transfer is
usually determined by observation.
AFFECTIVE OUTCOMES
 These are outcomes related to reaction,
motivation & attitude.
 One type of affective outcome is trainee’s
reactions towards the training program.
 Reaction outcomes measure the trainees response
to a training event such as training facilities,
trainer performance etc. They are also known as
trainer evaluation.
 Reactions are useful for identifying what trainees
thought was successful or what inhibited learning.
 Reaction outcomes are level 1 criteria in
Kirkpatrick's framework.
CONTD..
 Reaction outcomes are typically collected via
questionnaire completed by trainees.
 Other affective outcomes that might be
collected in an evaluation include tolerance
for diversity, motivation to learn, safety
attitudes & customer service orientation.
 Affective outcomes can be measured using
surveys.
 Affective outcomes relate to Kirkpatrick's
level 2 ( learning) or level 3 ( behavior)
depending on how they are evaluated.
RESULTS
 Results refer to overall benefits of training
for organization. Result outcomes include
improved safety at work site, increased
production, decreased in customer
complaints etc.
 Results outcomes are level 4 (results) criteria
in Kirkpatrick's framework.
RETURN ON INVESTMENT (ROI)
 ROI refers to comparing the training’s monetary benefits
with the cost of the training.
 ROI is considered outside of Kirkpatrick's four level & is
often referred to as level 5 evaluation.
 Training costs can be direct & indirect
 Direct costs include salaries & benefits for all employees
involved in training , instructors, consultants, program
material & supplies etc.
 Indirect costs are not related directly to the design ,
development or delivery of the training program.
 Benefits are the value that the company gains from
training program.ie; reduced costs or increased revenues.
 In many cases ,benefits can only be estimated.
PHASES OF EVALUATION
 PRE-TRAINING EVALUATION
 Identification of training needs
 Evaluate performance standards of the
trainee
 Evaluate training objective
 Input evaluation
 EVALUATION DURING TRAINING
 Observation
 Behavior analysis
 Course audits
 Session assessments
CONTD..
 POST-TRAINING EVALUATION
 Reaction evaluation
 Learning evaluation
IN DETAIL
Identification of training needs:
 if the line manager is actually interested & owns the
responsibility of developing his staff, he would
monitor their performance & identify their strengths
& weaknesses.
 If the deficiencies demand immediate correction,
then he may not even wait to respond to his training
needs.
 Sometimes training needs get identified when the
organization has to respond to
 Declining company image
 Change in technology/product/process
 Introduction of new system/policies/procedures
CONTD..
Evaluate performance standards :
 Directly linked to training needs is the
standards of performance.
 This information helps in setting realistic
objectives.
 While needs of the trainee are person
specific ,performance standards are job
specific.
 Performance standards identified as KRAs
finalized between the individual & his boss.
CONTD..
 Evaluate training objectives: this is to assess
whether there is a goal congruence between the
trainee's needs & training objective.
 Evaluate trainee’s profile: measuring pre training
knowledge, skills & attitude is desirable to
compare them with post-training performance.
 Input evaluation: input evaluation gives an
opportunity to the trainer to put things right
before the package becomes fully operational.
Committee approach has been found to be fairly
successful for input evaluation. Brainstorming
session would also help in situations where
feedback on the subject is already available.
CONTD..
 Observation: trainer’s perceptiveness &
observation skills cannot be discounted.
 Behavior analysis: usually assessment during
training through behavior analysis is relevant
for imbibing of certain skills & attitude for
example leadership skills, team orientation &
attitude towards learners.
 Course –audit: audits can be conducted to give
feedback for immediate course correction
 At the end of each day
 Mid way through the course
CONTD..
 Session assessment : Assessing each session
at the end or assessing each session at the
end of the day is possible. Normally this is
done by using semantic differential scale,
three point or five point.
 Reaction evaluation : at the end of the
course ,course reactions are collected by the
use of rating scale. Rating scale are forms on
which at the end each training session/
programme, trainees are asked to place on a
umber of five point scale.
CONTD..
 Learning evaluation: the purpose of evaluation at
this stage is to obtain information on the amount
of learning acquired during the training
programme, irrespective of whether they go on to
apply the learning on the job.
 Knowledge learning: the purest form of evaluation
of the knowledge learning is through tests
conducted at each stages-multiple choice,
objective tests or true or false questions. For open
ended training (objectives not formulated as
measurable),the best way to assess knowledge
changes is to ask trainees whether their knowledge
has improved .
CONTD..
 Skill learning: in most forms of the industrial
training , skill objective are more important
than knowledge objective. Since skills are
acquired by actual practice , they can be
best be evaluated by observing & analyzing
actual performance of the trainees while
they are practicing.
 Attitude learning : a common way of
evaluating attitude & opinions is to hand out
a questionnaire at the start & at the end of
the programme.

The Evaluation of Criteria & Phases of Evaluation

Uploaded by

Copyright:

Available Formats

The Evaluation of Criteria & Phases of Evaluation

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

The Evaluation of Criteria & Phases of Evaluation

Uploaded by

Copyright:

Available Formats

THE EVALUATION

Jose Paul Joseph

3 behavior Improvement of behavior on the job

2 learning Acquisition of knowledge, skills,

1 reactions Trainee satisfaction

You might also like