1

MODULE 4 Factors in Constructing Evaluative Instruments

OVERVIEW Measure evaluation developed these tools with the goal of maximizing program
results through the systematic collection and analysis of information and evidence about
health program performance and impact. However there are factors need to be
considered in giving an evaluation to our learners which need to be refine and identify
these factors in order to have good evaluation instrument.
In this module, we will discuss the evaluation instrument which are very mush helpful in
assessing students’ knowledge and development. Most evaluations require the use of a
data collection tool—a survey or other data collection instrument. Evaluators either need
to adopt or adapt tools “off the shelf” or create new ones.
LEARNING At the end of the lesson you are expected to:
OBJECTIVES  Discuss the different types of evaluation instrument
 Identify the factors in constructing evaluative instruments

Write your idea about the evaluation instrument. Answer the following questions:
 List down all types of tests you encountered
What do you
 Among the types of tests, which do you think is the easiest one? Explain
already know? your answer.
 Have you ever asked yourself how your teacher constructed your
examination (Social Science/Araling Panlipunan)?
Let’s get to Factors in Constructing Evaluation Instrument
Instruments are created for a particular audience. If your audience matches
know more the one that an instrument was designed for, great. If not, you’ll need to do some
testing to see if the instrument works for your audience before you use it for an
evaluation. For instance, a survey created for adults may or may not be
appropriate for children. You won’t know until you test it.
In a nutshell, assessment is a process, a tool for decision making; can be
applied to both individual and group; and is generated through product or
performance.

Type of measuring instrument

Measurement instruments are the central tools to acquire sound and
scientifically based knowledge and to make theoretical progress in a variety of
social scientific disciplines.
The types of decisions for which a measurement instrument is to be used
must be known before an instrument that yields dependable observations can be
designed. Two major types of decisions have to be distinguished. The first
concerns decisions that rely on the relative ranking of individuals. Accepting the
three top-scoring applicants for a position is an example of a decision that relies
on the relative interpretation of scores. Measurements that emphasize the
interpretation of the relative differences between scores are the domain of
classical test theory with its traditional focus on inter individual differences.
 2

The second major type of decision is based on the interpretation of the

absolute level of scores. Mastery tests in education provide an example of a
decision based on the absolute interpretation of a score. Similarly, pre-
established admissions standards for college (e.g., minimum SAT score) or
pass/fail standards on job selection tests rely on the interpretation of absolute
score levels.

Length of Instrument
The reliability of a measuring instrument can be increased almost indefinitely
(Spearman-Brown formula) by the addition of new questions equivalent to those
constituting the original instrument.
The Spearman-Brown Formula (also called the Spearman-Brown Prophecy
Formula) is a measure of test reliability. It’s usually used when the length of a test
is changed and you want to see if reliability has increased.
The formula is:

rkk = k(r11) / [1 + (k – 1)* r11]

Where:

rkk = reliability of a test “k” times as long as the original test,

r11 = reliability of the original test(e.g. Cronbach’s Alpha),
k = factor by which the length of the test is changed.
To find k, divide the number of items on the original test by the number of items
on the new test. If you had 10 items on the original and 20 on the new, k would
be 20/10 = 2.

Example question: a test made up of 12 items has a reliability (R11) of .68. If the
number of items is doubled to 24, will the reliability of the test improve?

Solution: Insert the given numbers into the formula and solve.
We are given:

r11 = .68.
k = 24/12 = 2.
So:
rkk = 2(.68) / [1 + (2 – 1)* .68] = .81.

Doubling the test increases the reliability from .68 to .81.

Validity- The extent to which the instrument really measures what it is intended to
measure.
 3

Reliability- The consistency with which an instrument measures a given variable.

Objectivity- The extent to which several independent and competent examiners

agree on what constitutes an acceptable level of performance.

Practicability- The overall simplicity of use of a test, both for test constructor and
for students.

Range of Difficulty
To determine the difficulty level of test items, a measure called the Difficulty
Index is used. This measure asks teachers to calculate the proportion of students
who answered the test item accurately.
Difficulty Index
It refers to the proportion of the number of students in the upper and lower groups
who answered an item correctly. The larger the proportion, the more students,
who have learned the subject are measured by the item. To compute the difficulty
index of an item,
the formula:
𝐷𝑖 =𝑛
𝑁
Where Di = difficulty index
n = number of the students selecting item correctly in the upper group
and in the lower group
N = total number of students who answered the test
Level of Difficulty
To determine the level of difficulty of an item, find first the difficulty index,
and identify the level of difficulty using the range given below:

Range of Difficulty index Interpretation/ Action

Difficulty Level
0.00 - 0.20 Very Difficult Revise or discard
0.21 - 0.40 Difficult Revise or discard
0.41 - 0.60 Right difficulty/Average Retain
0.61 - 0.80 Easy Revise or discard
0.81 - 1.00 Very Easy Revise or discard

Time Limits
Time limit on a test can be set to control the probability of a test taker
running out of time before completing it. The probability is derived from the item
parameters in the lognormal model for response times. Examples of curves
representing the probability of running out of time on a test with given parameters
as a function of the time limit are presented. Unlike the traditional methods of
dealing with test speediness, which assess the degree of speediness’ after the
test has been administered, the curves enables us to set a desired degree in
advance. The method is demonstrated using an empirical data set.
 4

Time Limit indicates that the assessment has a timer that will count down.
Once the timer reaches 00:00, you will not be able to answer any additional
questions. Some assessments are designed to see how many questions you can
answer within a specific time frame therefore you might not be able to answer all
of the questions within that time limit. Please try to work as accurately and quickly
as possible.

Determining the number of test items depend on the assessment format

you will be using in the examination. Aside from the difficulty of an item, the
number of hours is important on how many items will be included in the
examination. Here are the suggestions the will guide you on the number of test
items considering the format ate average time the student answer the
examination.
Assessment Format Average Time to Answer
True-False 30 seconds
Multiple choice 60 seconds
Multiple Choice of HOTS 90 seconds
Short Answer 120 seconds
Completion Type 60 seconds
Matching Type 30 seconds per item
Short Essay 10-15 minutes
Extended Essay 30 minutes
Visual Image 30 minutes
The number of items included in a given assessment will also depend on
the length of the class period and the type of items utilized.
It must be noted in testing elementary pupils, the testing time should be no more
than20 to 30 minutes so that proper motivation is maintained.
In high school and college levels, students can be given tests lasting several
hours – final examination, but most tests are limited to a testing time of 40 to 50
minutes because of the length of the typical class period.
In estimating the number of items to be used, keep in mind the slower students.

Objectives to be Assessed
Aligning assessment with the learning objective is of benefit to both
students and faculty. Students should be assessed on what they are taught and
those assessments measure the degree of learning. When reviewing the
assessments, faculty can readily see whether the students have successfully met
the stated objectives. Assessment can be built into the instruction and be an
integral part of the classroom activities.

There are three types of student learning objectives/expected outcomes:

Cognitive objectives—What do you want your graduates to know? (Terms,
concepts, facts, theories, principles, and methods, etc.)
Behavioral objectives—What do you want your graduates to be able to do?
(Written and oral communication; problem-solving; computational, leadership,
teamwork, and presentational skills, etc.)
 5

Affective objectives—What do you want your graduates to value or care

about? (appreciation for music, literature, and diversity; religious values; political
Awareness; ethnical awareness; and commitment to lifelong learning, etc.
Bloom’s Taxonomy is a well-known description of levels of educational objectives.
It may be useful to consider this taxonomy when defining your objectives.
Student learning objectives/expected outcomes should be worded so that
someone outside the discipline can understand them. Student Learning
objectives/expected outcomes should be S.M.A.R.T.—specific, measurable,
action oriented, reasonable, and time-specific.

Arrangement of Items
The use of multiple versions of objective exams which vary in test question
sequences has raised concern that such a practice may unconsciously bias
students' examination scores. Using effective of question sequencing on exam
performance of students as it related to memory factors. Arrangement of item is
found to have significant effective examination performance.
Arranging items should be arranged so that all items of the same type are
grouped together. The items should be arranged in order of increasing difficulty.
For some purposes, it may be desirable to group together items which measure
the same learning outcomes or the same subject matter content.

Scoring Procedure
Scored procedure means a written test, structured oral interview,
performance test, or other selection procedure or a combination of these
procedures that results in a numerical score to which percentage points may be
added.
Scoring key is a prepared list of answers to a given set of objective-type
questions. Suppose there are 10 multiple-choice objective type questions (each
having four options, A, B, C, D) in a section of the test, the scoring-key will be as
follows:
Section-I Scoring key
Q.N. 1 2 3 4 5 6 7 8 9 10
Key D C C A B D B A C B
A scoring key is prepared by listing serially the key (or right answer) to each
question against each item. For short answer type questions and essay type
questions, marking schemes are to be prepared (i.e., marks allotted to different
parts of the answer or to different important points etc. are to be mentioned). Such
scoring keys and marking schemes must be carefully prepared. They serve as
guides at the time of scoring the test and they ensure objectivity in scoring.

Method of Recording and Reporting Assessment Results

Summarizing success and Providing useful Progress feedback. In planning,
Assessment should be incorporated and evident in our Forward Plans. We need
 6

to plan for groups and where appropriate individuals. We need to plan effective
tasks and flexible teaching methods that maintain the correct balance across the
curriculum. We should strive to design tasks that aid progress and are
challenging, yet attainable. Teaching is the second stage of the assessment
process and is based on the planning already done. Teachers should encourage
a problem solving approach and create an atmosphere which promotes the
exploration of new ideas and activities. Teachers should use a variety of teaching
and learning approaches to meet the needs of all pupils. Our teachers’ forward
plans/programmes of study can serve a dual purpose as plans and records.
Recording will enable teachers to share with pupils’ successful learning and
identify development needs and next steps. It will monitor the effectiveness of
teaching and pupils’ progress in relation to attainment outcomes and targets. It
will enable teachers to report to parents and other teachers. It will also inform the
Head Teacher about attainment levels in classwork.

What you have read it just a glimpse about the idea on factors in constructing
evaluative instrument. Let us put your knowledge in practical knowledge in knowing how
this factors is vital in constructing an evaluation.
Let’s do and Provide a copy of exam sample of the teacher in social studies. Chose two type
exam method by the teacher and criticize his/her exam base on instruction, arrangement,
discover
and difficulty of item. If you choose Multiple choice as sample of criticism, make sure you
identify those item from simplest to difficult item and analyze those choice of their level
of difficulty. Provide only 10 item in the sample exam. Put in clean sheet of paper your
answer.

Accomplished the Worksheet No.1 Factors in constructing evaluative instrument

The activity will evaluate if you will understood the lesson.
How much you
have learned? The following criteria may be considered in checking the output:
1. Content of the answers is well organize
2. Correctness in analization
3. Timeliness of the submission
4. Technicalities (spelling, punctuation mark etc...)
Considering the value of evaluation instrument for improvement for teacher,
instructor, professor and school manager it is vital to know the factors in constructing
evaluative instrument in order to have good measuring instrument in our learners. This
Summary factor are so important in bring a good evaluation instrument in our learners. Providing a
good evaluation instrument it make our teaching style is effective and the goal of our
lesson will be achieve. It may be difficult to prepare an instruction for evaluation but this
is the only why see the effectiveness in our teaching style.
 March, Colin. Teaching Social Studies. National Library of Australia, Prentice-Hall
of Australia
Required  Calmorin L. P. Measurement and Evaluation, Third Edition. National Book store.
Readings Mandaluyong City 1550.
 7

 Calmorin L. P. (2011), Assessment of Student Learning 1.Firts Edition. Rex Book

Store, Inc
 De Guzman-Santos, R. (2007). Assessment of learning 1. Quezon City: Lorimar
Publishing, Inc.
 De Guzman-Santos, R. (2007). Advanced methods in educational assessment and
evaluation assessment of learning 2. Quezon City: Lorimar Publishing, Inc.
 https://www.slideshare.net/MadJutt/recordind-and-reporting-assessment-results-1
 Scored procedure definition - Law Insider
References  https://onlinelibrary.wiley.com/doi/pdf/10.1002/j.2333-8504.1982.tb01316.x
 http://egyankosh.ac.in/bitstream/123456789/42094/1/Unit-3.pdf
 what is Spearman-Brown formula - Google Search
 Measurement and Evaluation in Education _ MA-Edu _ ED-804 E _
English_21072017.pdf
 Educational measurement and evaluation

Feedback
 8

Factors in Construction
Evaluative Instrument

Evaluate the sample test below and identify the level of difficulty of item in every
questions. Explain each item why this choice is the correct answer and effectively of
distractors.
MULTIPLE CHOICE TEST
1. The Adventure of Tom Sawyer and The Adventure of Huckleberry Finn were
written by American author Samuel Langhorne Clemens. By what name is he better
known as?
A. Mark Twain C. Ernest Hemingway
B. Miguel de Cervantes D. Robinson Crusoe

2. Born in Sta. Cruz, Manila on August 2, 1799; he was the parish priest of Bacoor,
Cavite at the time he was involved in the Cavite Mutiny of 1872.
A. Andres Urdanete C. Jacinto Zamora
B. Vicente Garcia D. Mariano Gomez

3. Despite of strong protestation from Mang Jose, the government took away his
private property for public use upon payment to him or just compensation. What
power of the State was involved in the particular case?
A. Police Power C. Power of Eminent Domain
B. Power of Taxation D. Absolute Power of the state