Test Construction, Item

Writing & Item Formats

Presented by : Aamina Mohiyuddin
MSc. Clinical Psychology (Sem 1)
Submitted to : Sudeep P. K
Assistant Professor, Dept. Clinical Psychology
Table of contents

TEST CONSTRUCTION 3

CHARECTERSTICS OF A TEST 5

GENERAL STEPS OF 16
CONSTRUCTION
ITEM WRITING 26

GUIDELINES FOR ITEM 56

WRITING
REFRENCES 59
TEST
CONSTRUCTION
Meaning of a TEST in Psychology
According to the dictionary test is defined as a:
 series of questions on the basis of which sum information is
sought.
In Psychology and education:
 A psychological test is a standardized procedure to measure
quantitatively or qualitatively, one or more than one aspect of
a trade by means of a sample of verbal or non-verbal
behavior.
 According to Aplan and Saccuzzo (2001), they defined "A
psychological test or educational test is a set of items
designed to measure characteristics of human beings that
pertain to behavior".
CHARECTERSTICS OF A TEST
Psychologists have taken pain over classifying tests from the point of view of
different criteria. A brief is as follows:-

A. On the basis of the criterion of administrative conditions

• Tests have been classified into two types on the basis of administrative
conditions :-
a) Individual Test
a) Group Test
a) INDIVIDUAL TEST

• Tests that are administered to one person at a time.

• Used by school psychologists and counselors to
motivate children and to observe how they respond.
• Some individually administered tests are given
orally, and they require the constant attention of the
examiner.
• Individual tests, in general, have two limitations:
i. time-consuming
ii. used only when a crucial decision is necessary.
Eg: Kohls Block Design Test
b) GROUP TEST

• Group tests are tests which can be used

among more than one person or in a
group at a time

• Bell Adjustment Inventory is an example

of the group test.

• Besides assessing adjustment, group tests

are adequate for
 measuring cognitive skills to survey the
achievements
 strengths and weaknesses of the
students in the classroom, etc.
B. On the basis of the criterion of scoring
● Based upon this criterion, tests are classified into two types-
a)Objective test
b)Subjective test.

a) OBJECTIVE TEST
● Those whose items are scored by competent examiners or observers in such
a way that no scope for subjective judgement or opinion exists and thus, the
scoring remains unambiguous.
● Tests having multiple-choice, true-false and matching items are usually
called objective tests.
b) SUBJECT TEST:
• Tests whose items are scored by the competent examiners or observers in a way
in which there exists some scope for subjective judgement and opinion.
• As a consequence, some elements of vagueness and ambiguity remain in their
scoring.
• These are also called essay tests.
• Such tests are intended to assess an examinee's ability;
 to organize a comprehensive answer
 recall and select important information
 present the same logically and effectively.
• Since in these tests the examinee is free to write and organize the answer, they
are also known as free-answer tests.
• The following items illustrate the nature of an essay test:
(i) What are the major goals of education?
C. On the basis of the criterion of the nature or contents of items
• A test may be classified on the basis of the nature of the items or the contents
used therein
• Important types of tests based on this criterion are;

(i) Verbal test

(ii) Nonverbal test
(iii) Performance test
(iv) Nonlanguage test
• (
(i) VERBAL TEST
• One whose items emphasize reading, writing and oral expression as the primary
mode of communication.
• Herein, instructions are printed or written.
• These are read by the examinees and, accordingly, items are answered.
Eg:- Jalota Group General Intelligence Test
Mehta Group Test of Intelligence

(ii) NONVERBAL TESTS

• Those that emphasize but don't altogether eliminate the role of language by using
symbolic materials like pictures, figures, etc.
• Test items present the problem with the help of figures and symbols.
• Nonverbal tests are commonly used with young children as an attempt to assess the
nonverbal aspects of intelligence such as spatial perception. Eg:-
Raven Progressive Matrices.
(iii) PERFORMANCE TEST

• Requires the examinees to perform a task rather than answer some questions.
• Such tests prohibit the use of language in items.
• Occasionally, oral language is used to give instruction, through gesture.
• Different kinds of performance tests are available.
• Some tests require examinees to assemble a puzzle, place pictures in a correct
sequence, place pages in the boards as rapidly as possible, point to a missing
part of the picture, etc.
• They are usually administered individually so that the examiner can count the
errors committed by the examinee.
• Their emphasis on the examinee's ability to perform a task rather than answer
some questions.
(iv) NONLANGUAGE TESTS

• Those which don't depend upon any form of written, spoken or reading
communication.
• Such tests remain completely independent of the ability to use language in
any way.
• Instructions are usually given through and the examinees respond by
pointing at or manipulating objects such as pictures, blocks, puzzles, etc.
• Such tests are usually administered to those persons or children who cant
communicate in any form of ordinary language
D. On the basis of the criterion of purpose or objective
• Tests are also classified in terms of their objectives or purposes.
• Based upon this criterion, tests are usually classified as
a) Intelligence Tests
b) Aptitude Tests
c) Personality Tests
d) Neuropsychological Tests
e) Achievement Tests.

Intelligence tests intend to assess intelligence of the examinees. Aptitude tests assess
potentials or aptitudes of the persons. Personality tests assess traits, adjustments,
interests, values, etc., of the persons. Neuropsychological tests are the tests, which are
used in the assessment of persons with known or suspected brain dysfunctioning.
Achievement tests assess what the persons have acquired in the given area as a
function of some training or learning.
E. On the basis of the criterion of standardization

• Tests are also classified on the basis of standardization.

• Standardized tests are those which have been subjected to the procedure of
standardization.

D. On the basis of the criteria of time limit in producing the response

• The tests are classified into
i. Power Test
ii. Speed Test

A power test is one which has a generous time limit so that most examines are able to
attempt every item. Speed test are those that have severe time limits but the items are
comparatively easy. Speed test reveal how rapidly with what speed the examine can
respond within a given time limit.
GENERAL STEPS OF TEST CONSTRUCTIONS
1. PLANNING OF THE TEST
2. WRITING ITEMS OF THE TEST
3. PRELIMINARY ADMINISTRATION OF THE TEST
4. RELIABILITY OF THE FINAL TEST
5. VALIDITY OF THE FINAL TEST
6. NORMS OF THE FINAL TEST
7. PREPARATION OF MANUAL AND REPRODUCTION
 1. PLANNING OF THE TEST
● The first step in the construction of a test is careful planning.
● Test construction specifies the;
 Broad And Specific Objectives Of The Test
 Decides Upon The Nature Of The Content
 The Type Of Instructions
 The Method Of Sampling
 A Detailed Arrangement Of The Administration
 The Final Administration Time Limit For The Completion Of The Test
 Statistical Methods To Be Adopted
 2. WRITING ITEMS OF THE TEST
• A lot depends upon the item writer's intuition, imagination, experience,
practice and ingenuity. However, there are some essential prerequisites,
which must be met if the item writer wants to write good and appropriate
items.
• These requirements are enumerated as follows:
1. The item writer must have a thorough knowledge and complete mastery
of the subject matter. He must be fully acquainted with all facts,
principles, misconceptions ,fallacies in a particular field so that he may
be able to write good and appropriate items.
2. The item writer must be fully aware of those persons for whom the test is meant.
He must be aware of the intelligence level of those persons so that he may
manipulate the difficulty level of the items for proper adjustment with their ability
level. He must also be able to avoid irrelevant clues to the correct responses
3. The item writer must be familiar with different types of items along with their
advantages and disadvantages.
4. The item writer must have a large vocabulary. He must know the different
meanings of a word so that confusion in writing the items may be avoided. He must
be able to convey the meaning of the items in the simplest possible language.
5. After writing down the items, they must be submitted to a group of subject
experts for criticism and suggestions
3. PRELIMINARY ADMINISTRATION OF THE
TEST
● When the items have been written down and modified in the light of the
suggestions and criticisms given by the experts, the test is said to be
ready for its experimental try-out.
● The main purpose of the experimental try-out of any psychological and
educational test is as given in the next slide:
1. Finding out the major weaknesses, omissions, ambiguities and inadequacies of
the items.
2. Determining the difficulty values of each item which, in turn, helps in selecting
items for their even and proper distribution in the final form.
3. Determining the validity of each individual item.
4. Determining a reasonable time limit of the test
5. Determining the appropriate length of the test. In other words, it helps in
determining the number of items to be included in the final form.
6. Determining the intercorrelations of items so that overlapping can be avoided.
7. Identifying any weakness and vagueness in directions or instructions of the
test as well as in the fore-exercises or sample questions of the test.
4. RELIABILITY OF THE FINAL TEST
● Reliability is the self-correlation of the test and it indicates the
consistency of the scores in the test.
● There are three common ways of calculating reliability coefficient,
namely test-retest method, split-half method, and the equivalent-
form method
5. VALIDITY OF THE FINAL TEST
● Validity refers to what the test measures and how well it measures.
● If a test measures a trait that it intends to measure well, we say that the
test is a valid one.
● After estimating the reliability coefficient of the test, the test constructor
validates the test against some outside independent criteria by comparing
the test with the criteria
● Validity may also be defined as the correlation of the test with some
outside independent criteria.
● Validity should be computed from the data obtained from the samples
other than those used in item analysis
6. NORMS OF THE FINAL TEST
• Norms are defined as the average performance or score of a large
sample representative of a specified population.
• Norms are prepared to meaningfully interpret the scores obtained on
the test for, as we know, the obtained scores on the test themselves
convey no meaning regarding the ability or trait being measured.
• But when these are compared with the norms, a meaningful inference
can immediately be drawn.
• The common types of norms are the age norms, the grade norms,
the percentile norms, and the standard score norms.
7. PREPARATION OF MANUAL AND
REPRODUCTION
• In the manual the test constructor reports the psychometric properties of the
test, norms and references.
• This gives a clear indication regarding the procedures of the test
administration, the scoring methods and time limits.
• It also includes instructions as well as the details of arrangement of materials,
that is, whether items have ben arranged in random order or in any other order.
• In general, the test manual should yield information about the standardization
sample, reliability, validity, scoring as well as practical considerations.
ITEM
WRITING
 MEANING AND TYPES OF
ITEMS
● Bean (1953, 15) defines item as "a single
task or question that usually cannot be
broken down into any smaller units".
● An item must have the following
characteristics:
1. An item should be phrased in such a manner that there is no ambiguity
regarding its meaning for both the item writer as well as the examinees
who take the test.
2. The item should not be too easy or too difficult.
3. It should have discriminating power, that is, it must clearly distinguish
between those who possess a trait and those who do not.
4. It should not be concerned with the trivial aspects of the subject matter.
5. As far as possible, it should not encourage guesswork by the subject.
6. It should not present any difficulty in reading.
7. An item writer must also have sufficient knowledge regarding the
different forms or types of items as well as the merits and demerits of
each type.
• In a broad sense, items are of two types:

1. Essay item or free-answer item

2. Objective item or 'new type' of
item
1. ESSAY ITEM OR FREE-ANSWER
ITEM
● An Essay Item is one in which the examinee relies upon his memory and
past associations to answer the questions in a few words only.
● Since such items can be answered in whatever manner one likes, they are
also known as free-answer items.
● Essay items are most appropriate for measuring higher mental processes
which involve the process of synthesis, analysis, evaluation, organization
and criticism of the events of the past. Essay tests are thus suitable for
measuring traits like critical thinking, originality and the ability to integrate
or synthesize or analyze different events
 Essay items are of two types:

A. Short-answer Type:
● The examinee supplies the answer in one or two lines and is usually
concerned with one central concept.
Eg:- Explain the meaning of reliability of an educational test.

A. Long-answer Type Or Extended-answer Essay Type:

● Where the examinee's answer comprises several sentences
Eg:-Describe the methods of estimating reliability and validity of an
educational test.
• Nowadays essay items are frequently used by teachers to measure the
achievement of pupils in a classroom.

• The primary merit of such items is that they encourage the examinee to give
a coherent and organized picture of his memory and past associations.

• They require the examinees to organize and produce the answer rather
than recognize the answer.

• A better inference can be drawn regarding the examinee on the basis of these
answers.
• The most serious drawback of such an item:

• It varies from scorer to scorer and sometimes within the same scorer, when
he is asked to evaluate the answer at different time intervals.

• Not only this, scoring of the essay items takes a longer time because of the
length of the answer and also because the scorer is required to read each line
very carefully. Thus essay items lack scoring economy.
 Essay test items present two important difficulties

 The factual knowledge may sometimes be confounded with the ability to

organize and synthesize an answer.

 The examinee who knows more fact correctly may write an answer that looks
better and superior when the originality may be missing from the answer.

 He is required to think out an answer and compose a written response.

Naturally, this takes more time.

 As a consequence, differences in performance can depend on ability to write

quickly in correct form..
To combat these problems, several variations in the testing environment
have been proposed by psychologists and educators.
• Following are the important ones:-
(i) Open-book examination:

• The examinees are permitted to use their textbooks or class notes for answering
the questions.
• This technique reduces the impact of differences in knowledge.
• The demerit of this method is that the less able examinees may devote much
time in searching material during the test whereas the more able examinees
devote much time in composing the answers.
• Thus, the examinees who devote more time in searching the materials in
books or notes may feel that they have less time than if the examination would
have been on the pattern of closed-book
(ii) Take-home examination:

• In this type of examination the examinees don't have time pressure during
the test and it allows the examinees to work in a completely unsupervised
situation as long as he or she wishes
• One major drawback of this format is that
 it sacrifices test security.
 There is no way to know that the examinee has worked independently.
(iii) Cheat sheets:

• Cheat sheet is one variation of open-book examination.

• Examine is allowed to bring to examination one piece of paper with as many
facts, procedures, points or other information as desired.
• This technique has been found to work well in courses where formulas or
several small points or facts constitute a large part of the subject matter.
• This technique reduces dependence on the textbooks.
(iv) Study questions:

• This technique combines features of both the open-book and take-home

examination.
• Examinee is provided with a list of study questions from which examination
questions will be set or drawn.
• For example, the examinee may be provided a list of 10 to 15 questions and
told that the examination will contain 4 or 5 of them.
• This technique has the advantage
 The examinees can take as much time as they wish for preparing for the
examination
 Can also use whatever resources are available to them intensively.
In order to write good essay items, the following suggestions must be kept in
mind:

1. An essay item must contain explicitly defined problems.

2. It must contain such problems whose answers are not very wide.

3. Essay items must have clear-cut directions or instructions for the

examinees.

4. Sufficient time should be allowed in the construction of the essay items.

5. One should start essay item/question with phrases such as 'compare',
'contrast',', etc. The use of these words or phrases will help present the
examinees with task requiring them to select, organize and apply
information.

6. The item writer must clearly know what mental processes he wants to asses.

7. It should be written in such a way that task is very clearly defined

2. OBJECTIVE ITEM OR
'NEW-TYPE' ITEM
● An objective or new-type item is one wherein there is only one fixed
correct answer, which either the examinee gives on his own or he is
required to select from among a given few.
● All objective items can be divided into two broad categories:

A. The Supply Type: One wherein the examinee is to write down the correct
answer on his own.
B. The Selection Type: A selection type of item is one wherein the
examinee is to select or identify the correct answer from a few given.
THE SUPPLY TYPE OF ITEM IS DIVIDED INTO TWO MAIN
CATEGORIES:

1. The Unstructured Short-answer Item:

● One which is given in a question form and where the examinee supplies
the correct answer in terms of a word, number or phrase.
2. A Completion Item Or Fill-in Item

● A completion item or fill-in item is one which is presented in the form of

an incomplete statement.
● The examinee is supposed to complete the statement by supplying the
missing word, number of phrase, etc.

Eg:
1. Who was the first president of the Indian Republic? (Dr Rajendra Prasad).
An Unstructured Short-answer Item
2. The first president of the Indian Republic was Dr Rajendra Prasad. A
Completion Or Fill-in Item.
● In writing the completion items, the item writer must keep in mind the
following points:

1. The item must not be ambiguous

Given here are two examples of how the same item can be worded in an
ambiguous form and a clear-cut form.
 Ambiguous form:
Behaviorism was founded in ______________

The above item has two possible answers: 1913' and 'America', both of which
are correct.
 Clear-cut form:

Behaviorism was founded in the year ________

or
Behaviorism was founded in the country known as __________
2. Clues which give a direct hint to the correct answer must be avoided.

For example:-
 A person who believes in God is known as ___________ and a person who
does not believe in God is known as __________

Obviously, in the first blank "theist" and in the second blank its opposite,
"atheist" will be filled in. In such items when the examinee knows the answer of
the first blank, he can, without much reasoning, answer the second blank because
it is the opposite of the answer of the first blank.
3. A large number of blanks should not be used in completion items because in
that case, originality (and not the real omissions) would be the only source of
response.

For example:
 Personality is the dynamic organization ________that _______ his ________

In the above completion item the examinee is required to give the definition of
personality,
• Selection-type items, as used in psychological, sociological and
educational research, are of many varieties.

Some of the important varieties are:

1. TWO-ALTERNATIVE
ITEMS
2. MULTIPLE-CHOICE
ITEMS
1. TWO-ALTERNATIVE ITEMS
● In this type of item, only two answers are provided from which the
examinee is required to select the one which he thinks to be correct.
● It is further divided into four subcategories.

(a) Yes-No:

● This is very common. Here items are expressed in the form of a question.
For example:
1. Do you suffer from insomnia? Yes/No
2. Do you feel shy in talking with the members of the opposite sex? Yes/No
 b) True-False:

• True-false form of objective item is expressed in the form of a which is either

entirely true or entirely false.

For example:
1. Wilhelm Wundt is regarded as the father of experimental psychology.
True/False
2. Adolf Adler was the father of psychoanalysis. True/False
c) Right-Wrong:

• Right-wrong form of objective item is expressed in terms of simple

sentences, which are to be marked as right or wrong,

For example:
1. He needs not go to cinema. Right/Wrong
2. Mohan said that the sun rose in the east. Right/Wrong
2. MULTIPLE-CHOICE ITEM
• It is the most popular, common, flexible and effective of all objective items.
• In each item of the multiple-choice, there is one stem in the form of a question
or incomplete statement, which is generally followed by a set of three to five
answers or options.
• The following examples illustrate a multiple-choice item:

The theory of psychoanalysis was advanced by:

(a) E L Thorndike
(b) I P Pavlov
(c) J B Watson
(d) S Freud
• Another advantage of multiple-choice item over two- alternative item is
that the respondents cannot receive credits for simply knowing that
the statement is incorrect, rather they must also know what is
correct.

• Look at the difference in the following two items:

Q. Mrs Indira Gandhi was assassinated in 1986. True/False

Q. Mrs Indira Gandhi was assassinated in:
a) 1984
b) 1986
c) 1994
d) 1985
3. MATCHING ITEMS.
● In matching item, there are two columns right and left. Match the
names with the statistical techniques:
GENERAL GUIDELINES FOR ITEM
WRITING
● Clarity in writing items is one of the main requirements of an item to
be considered good.
● Items must not be written as verbal puzzles. They must be able to
discriminate between those who are competent, and those were not.
● Nonfunctional words must not be included in the item as it tend to
lower the validity of the item.
● Item writer must make sure that irrelevant and intentional
incorporation of the items must be avoided.
● The items must not be too easy or too difficult for the examines.
● Irrelevant clues must be avoided.
REFRENCES
• Singh, A. K. (1986). Tests, measurements and research methods in
behavioural sciences.
• Domino, g., & Domino, M. L. (2006). Psychological testing: an
introduction. Cambridge university press.
• Test screening IQ - Kohs. (n.d.). YouTube.
https://www.google.co.uk/url?sa=i&url=https%3A%2F%2Fwww.yout
ube.com%2Fwatch%3Fv%3Dxze_9ION-4I&psig
• Redirect notice. (n.d.). https://www.google.co.uk/url?sa=i&url=https
%3A%2F%2Fstudy.com%2Facademy%2Flesson%2Ftest-
construction-item-writing-item-analysis.html&psig
THANK
YOU