Planning and Designing

Assessments
Henry C. Encabo
College of Education
10.13.2022

University of Southeastern Philippines

In this presentation

1. A Review on Making a Test Specifications

2. Designing Assessments with Diagnostic Value
University of Southeastern Philippines
University of Southeastern Philippines
University of Southeastern Philippines
University of Southeastern Philippines
Test Development Process
Unpacking of
Learning Making a Test
Item Writing
Competencies Specifications

Test Administration Item Analysis Item Banking

University of Southeastern Philippines

The Test Specifications

University of Southeastern Philippines

 Some Considerations
★ Which competencies should be assessed by a written test?
★ Can a competency be assessed using items from different
cognitive levels?
★ How should the items be distributed?

University of Southeastern Philippines

Which competencies should be
assessed using a written test?
Evaluate narratives based on how the author developed the
elements

Identify the purpose, key structural and language features of

various types of informational/factual text

Compose clear and coherent sentences using appropriate

grammatical structures (verb tenses, conjunctions, adverbs)

University of Southeastern Philippines

 Unpacking Complex Competencies
to Clarify what Items to write
Identify the purpose, key structural and language features of various types of
informational/factual text
Identify the purpose of an informational/factual text

Identify the text structures in an informational/factual text

Identify signal words associated with each type of information/factual text

University of Southeastern Philippines

 Unpacking Performance-based
Competencies to make a test item
Compose clear and coherent sentences using appropriate grammatical structures
(verb tenses, conjunctions, adverbs)
Determine sentences that support or are coherent with a given sentence
Determine the appropriate verb tenses in a sentence
Use the appropriate conjunctions to complete a paragraph

University of Southeastern Philippines

Can a competency be assessed using
items from different cognitive levels?

YES but Unpack.

Some competencies are complex and needs to be stated in unpacked statements

University of Southeastern Philippines

Can a competency be assessed using
items from different cognitive levels?

Remembering Understanding Applying Analyzing

Competency 1 1,2 3 4

Poorly Designed Specifications

University of Southeastern Philippines

Can a competency be assessed using
items from different cognitive levels?
Remembering Understanding Applying Analyzing
Competency 1
Unpacked Statement I 1,2
Unpacked Statement 2 3
Unpacked Statement 3 4

Better Designed Specifications

University of Southeastern Philippines
How should the items be
distributed?

★ DepEd Order #79 s. 2003 (60 - 30 - 10)

★ Using the complexity required by the competencies

University of Southeastern Philippines

Understanding Item/Test Difficulty
with the Rasch Model

“ The difficulty of an
item is relative to a
person's ability “

University of Southeastern Philippines

 Persons Items
Note: More persons with Henry has < 50%
higher cognitive levels chance on these items

Henry has 50% chance

on this item

Henry
Henry has > 50%
chance on these items

Note: More items with

lower cognitive levels

University of Southeastern Philippines

 The competency
statements reveal
the expected
cognitive demand

The more lower level

competencies you target, the
lower the cognitive demand
levels would be
University of Southeastern Philippines
Designing Multiple Choice Tests with
Diagnostic Value
 www.usep.edu.ph

For Every Response is a

Cognitive Rule
Cognitive Rules are social constructs that convey beliefs and
reasons for thought. They do not have to be “correct” but
maybe believed to be so.
 www.usep.edu.ph

Interpretability of Cognitive Rules

D
 www.usep.edu.ph

2 Add the numerators and

𝑎+ 𝑏
5 denominators 𝑐+ 𝑑
1 1
+ =? 5 Find the LCD and convert 𝑎×𝑛 𝑏 ×𝑚
2 3 6 to similar fractions then
add
+
𝑐 ×𝑛 𝑑 × 𝑚

Cross-multiply and (𝑎 × 𝑑)+(𝑏 ×𝑐)

add 𝑐×𝑑

3
No Rule !!!
10
 www.usep.edu.ph

A Design for a Diagnostic MCQ

Interpretable Response
Task
Level 1 Cognitive Rule
Mother Stem: Main
Problem or Text Level 2 Cognitive Rule
Level 3 Cognitive Rule
Level 4 (Correct) Rule
Stem: Questions/
Items
 www.usep.edu.ph

A Taxonomy of Option Rationale (Based from King., et al, 2004)

Conceptual Errors and

Level 1 Misunderstandings

Level 2 Initial Understanding

Weak or Inconsistent
Level 3
Reasoning

Level 4 Correct and complete answer

 www.usep.edu.ph

Conceptual Errors and

Level 1
Misunderstandings

• Commits errors in decoding, retrieving

and processing information (from the
tasks)
• Answers using some prior knowledge
unrelated to the problem
• Implements strategies & reasoning not
related to the task
 www.usep.edu.ph

Level 2 Initial Understanding

• May correctly understand the given

information but tends to be too literal or
focus superficial connections
• Knows the operations to solve a problem,
but does not have a clear framework to
address it (as a result, implements a
flawed strategy)
• Misapplies a valid framework of reasoning
 www.usep.edu.ph

Level 3 Weak or Inconsistent Reasoning

• Uses a correct method or framework of

reasoning but comes up with a weak or
incomplete answer
• Responses are too narrow or too broad
• Uses inconsistent reasoning that results
to “almost” having the answer correct
• Tends to commit errors by being careless
or being inconsistent
 www.usep.edu.ph

Level 4 Correct and Complete answer

www.usep.edu.ph
 www.usep.edu.ph

Best Answer Variety !!!

www.usep.edu.ph
www.usep.edu.ph
 www.usep.edu.ph

Response Level and Differentiated Instruction

Conceptual Errors and Misunderstandings Re-teaching/ Re-learning

Level 1

Guided Instruction/ Reciprocal or

Initial Understanding
Level 2 Collaborative Learning

Level 3 Weak or Inconsistent Reasoning Drills, Practice and Feedback

Self-paced learning activities

Level 4 Correct and complete answer
 www.usep.edu.ph

Tips for Better Stem

The stem should present a definite problem and be free from irrelevant information
 www.usep.edu.ph

Tips for Better Stem

Use NEGATIVE words only if necessitated by an outcome. Put the negative word in the
stem and emphasize it by underlining, italics or bold
 www.usep.edu.ph

Tips for Better Options

The relative length of the options should NOT provide a clue to the answer
 www.usep.edu.ph

Tips for Better Options

Do not use “All of the above” and “None of the above”
 www.usep.edu.ph

Tips for Better Options

Arrange the options in a logical order
Got
Questions?
 GROUPINGS

Practical Research 2 Personal Development Intro to World Religions &

Patricia P. Baan Marian Ivy B. Bacalla Belief Systems
Moises Baula Joralyn T. Vivas Mitchille A. Alanza
Marjorie Pagente
21st Century Literature from the Phil. and
the World
Phil. Politics
Angelica H. Belamide EIM Janine P. Canillas
Michael R. Navarro
UCSP MIL
Mirasol R. Pantinople Oral Communication Rannie Jhon D. Taborada
Beverly L. Estabaya Mary Grace L. Tajores

Introduction to the Philosophy of the SMAW

Human Person Michael V Chavez
Chael D. Dagatan Alfredo D. Satumacacal Jr
Maryan Novah O. Ejusa Daniel M. Dayupay
 ACTIVITY

Craft your TOS and 2 Test Questions

(Items)
Practical Research 2 Applying/Analyzing Intro to World Religions & Belief Systems

Phil. Politics
21st Century Literature from the Phil. and Applying/ Analyzing
the World MIL

UCSP Evaluating/Creating

Introduction to the Philosophy of the Evaluating/Creating

Human Person

Personal Development Evaluating/Creating

EIM
Applying/Analyzing

Oral Communication Evaluating/Creating

Applying/ Analyzing
SMAW