Question Preparation, Validation
Question Preparation, Validation
Question Preparation, Validation
ON
QUESTION BANK
PREPARATION,
VALIDATION AND
MODERATION BY
PANEL
SUBMITTED TO:
PROF.K.CHANDRALEKHA
DEPT. OF MEDICAL SURGICAL NURSING
ICON
SUBMITTED BY:
MR.P.NEELAKANDAN
M.SC NURSING I YEAR
ICON.
SUBMITTED ON:
QUESTION BANK-PREPARATION VALIDATION,
MODERATION BY PANEL
INTRODUCTION:
The question bankmakesavailablestatisticallysoundquestions of known technical
worth and model
questionpapersandthusfacilitatesselectionofproperquestionsforawelldesigned to
questionpaper. A large number of questions can be created and pooled from which the faculty
can select appropriate questions based on the test blueprint. Test banks also are created easily
by using a word processing program that is used to store test items and make item and make
item revision easier. All the questions would be vetted by course team members for quality
and then they would be indexed and banked. These questions when pooled in a bank, is called
a question bank.
The advantage of question bank is that questions which work out well in practice can
be reused on a number of later situations. Thus, new questions do not have to be generated at
the same rate from year to year and the quality of question gradually improves. The question
bank is, thus a planned library of test items pooled through the cooperative efforts under the
aegis of an institution for the use of evaluators, academics and students in partial fulfilment of
the requirements of the teaching-learning process.
DEFINITION:
“Arelativelylargecollectionofeasilyaccessibletestquestions.”
Thequestionbankmaybedefinedasakindofreservoirofanumberofsetsofquestionson
Eachsubjectinwhichexaminationistobeheldandfrom whichasetforany particular
examinationcouldbepickedout at random andatshortnoticeandsenttothepress.
Itisaplannedlibraryoftestitemsdesignedtofulfilcertainpredeterminedpurposes.It should
covertheentireprescribedtext.
PRINCIPLES:
Bank planning: Analysis of subject matter and content
Collection of test items: Teachers and item writers specially trained for purposes, Past
Examination papers
Try out and item analysis
Using Item analysis data
Banking selected items
Administering Sample test
III.COURSEELEMENTS:
Grading
Examinations
Textbook
Assignments
Audio-Visualaids
Technologyusage
Coursedifficulty,Paceandworkload
LEVELSOFQUESTIONBANK:
ZeroLevel:Thequestionbankisjustquestionsandquestionsareclassifiedaccordingthe
areasofthesyllabus
LevelOne:Certaindetailsarrivedintheform ofguessestimatesbyconsensusof
Experiencedteachersandsubjectmatterexperts
Leveltwo:Thequestionsareclassifiedaccording tothecontentoflearningobjectivesthat
they testandeachquestionispre-testedanditem analysiscarriedouttogivemoreaccurate
Informationsuchasthefacilityindexanddiscriminationindex
Levelthree:Itisamereextensionofleveltwo, atthislevelthequestionswiththeirtechnical
detailsarestoredinacomputerfacilitatingtheirretrievalandmanipulationwithinavery short time.
WRITING TEST DIRECTIONS:
The written test directions should be self-explanatory. It should include the following
information:
Time allotted to complete the test
Instruction for responding (choose the most appropriate answer)
Instruction for recording the answer in answer sheet
Marks assigned to each question.
REPRODUCING THE TEST:
o The test should be easy to read and follow. The following guidelines are suggested:
o Type test neatly
o Space items evenly
o Number items consecutively
o Keep item stem and options on the same page
o Place introductory material (graph or chart) before item
o Keep matching on the same page
o Proofread after compiling and before duplicating
ADVANTAGES:
It is the storage of large number of question.
It saves time and energy over conventional testdevelopment.
It provides platform for discussing curriculum goals and objectives.
Questionbankmakeavailablereadymadetestitemsforusebyeveryteacher
Thecooperativeeffortsresultintheimprovementofitem quality
Mostoftheexaminationweaknessesareminimizedbyusingquestionbanks
DISADVANTAGES:
Questionbankisnotcureallformeasurementproblems
It requires great deal of time in preparation, planning and development of the question
bank.
All the items should be analysed before including in the question bank.
Item analysis involves the use of various mathematical and statistical procedures.
GRADES:
In the evaluation system, grading is a recent phenomenon, earlier and even now in
many course of study the scoring system is still used. Grading when compared to
traditional system of scoring has some pertinent advantage.
Drawbacks of traditional scoring system:
Marking involves subjectivity and bias
Results are declared as either or fail
All scores have to be summated the end for assigning a particular division
The traditional grading system was to assign a single letter grade for each subject-
A,B,C,D,E
BASIC DRAWBACKS OF THE TRADITIONAL GRADING SYSTEM:
They are a combination of achievement, effort, work habits and good
behavior
The proportion of students assigned each letter grade varies from teacher to
teacher.
They do not indicate the student’s specific strength and weakness in learning
While assigning grades the following queries have to be classified.
What should we include in letter grade?
How do we convert a client data into grades?
How do we create a frame for grading?
Basically, letter grades are meaningful if they represent achievement, but if it is mixed
up with the other external factors like efforts, works etc, it gets contaminated as they do not
consider quality aspects.
However, teachers feel even these factors should be considered while giving grades.
It is very difficult to assess exactly a student’s effort/potential; also it is hard to distinguish
between aptitude and achievement.
Some students are good at certain aspects and other are good in some other aspects.
This means if grading is considered differently for different students if may send wrong
messages that may be unfair at times.
While converting scores into grades different components have to be given different
weightages and the composite score should be generated for which grade should be decided.
For instance, 60% weightage to the final examination, 20% to the presentation and
20% to be decided and together the composite should be generated for 100.
But in the above example, the weightage given to different components being
different the strengths of the student in each area may vary and affect their aggregate score.
The other way may be to take less number of aspects, e.g. 2 aspects or assign equal
weightage to different components, thus the scoring and grading becomes more objective. In
fact a more refined weightage could be using standard deviation as a measure of variability.
If teachers are equipped with computing skills then measurement becomes very simple and
more objective
According to Jacobs and Chase (1992), the characteristics of a good grading system
include:
Informing students of the specific grading criteria at the beginning of the course
(stated clearly in the prospectus/syllabus).
Grades should be assigned only based on the learning outcomes and not taking
into considerations of the factors such as attendance and effort.
Collecting sufficient data before assigning a grade
Recording data collected for grading purpose quantitatively (e.g. 90% not A)
Following uniform grading systems for all students
Using statistically sound principles for assigning grades
ASSIGNING GRADES:
Grading the students not only gives feedback and it also motivates the students. Each
institution might have grading policy or scale. The two basic methods for assignment the
absolute and the relative or comparative scales.
ABSOLUTE SCALE:
The grading is very convenient when the course objective have been clearly specified
and standard and mastery level approximately set the letter grades in an absolute system may
be defined as the degree to which objective have been attained. The students’ earned marks in
percentage are compared with the standard and grades are assigned
They can be discussed in the following ways
Pre-established percentage score
Criterion referred grading
Numeral rating
I. PRE-ESTABLISHED PERCENTAGE SCORE:
The different range of percentage marks for assigning grade letter is used. The test
and the assessment designed to yield scores in terms of percentage of correct answer; the
absolute grading can be decided as given below:
A =95% to 100% correct
B =85% to 94%
C =75% to 84%
D =65% to 74%
E =below 65% correct
For example:
Assigned grades
90-100 A
80-89 B
70-79 C
60-69 D
<60 E
II. CRITERION REFERRED GRADING:
The criterion referenced standard has been fixed either by the teacher or the
authorities before hand in view of the difficulty the test, and standard or quality of learning
performance needed from learners, i.e. decision with regard to the students performance in
terms of behavioural changes, on the basis of which letter grades are gained.
This can be represented as:
Grade Performance Level of performance
A Outstanding Students has mastered all the courses, major and minor
instructional goals
B Very good Has mastered all the course, major instructional goals
and most of the minor ones.
C Satisfactory Student has mastered all the courses, major
instructional goals but just a few minor ones.
D Very weak Student has mastered just a few of the courses major
senior instructional goals are basically has essential
needed for the next highest level of instruction.
E Unsatisfactory Student has not mastered any of the lower major
instructional goals. Does not need next higher level
instruction. Remedial work is needed
LIMITATIONS:
o Teacher have no flexibility to prepare as per the local needs
o Objective based performance may not be seen all the times.
RELATIVE SCALE:
o A relative scale rates students according to their ranking within the group.
o The faculty has to record the scores of all students in a descending order.
o Grades may then be assigned using a variety of techniques.
o One method is to assign the grades using natural “breaks” in the distribution. This
method has disadvantage of being subjective.
o The other method is to find out the measures of central tendency (mean or mode). In a
bell shaped curve (normal distribution) mean and mode will be the same and in
skewed distribution, median can be used as a measure of tendency.
o Then determine the standard deviation. The C grade will be set as the mean plus or
minus one half the SD (encompassing 40% of the scores).
Grade Calculation
A >upper B
B Upper + 1SD
C Mean ± 5 SD
D Lower – 1 SD
E < Lower D
EXTERNAL MODERATION
External moderation is undertaken by the external examiner, who will inform the
program leader of the procedures that he or she would like adopt for this.
Normally the external examiner would receive all referred papers and all papers
awarded the highest grade, as well as a sample of papers from each grade or
percentage band. The programme leader needs to ensure that the external examiner
has a time scale sufficient for adequate scrutiny of the papers.
Once the moderation system has been completed, assessment work, can be returned to
students. Assessments are made available for collection at the departmental office.
Students should note that any mark awarded is provisional at this stage. The final
mark is determined when the board of examiners meets, and a unit pass list is sent to
each successful student as soon as possible after that meeting. Students who were
unsuccessful are informed individually by letter.
PURPOSE OF MODERATION
The purpose of moderation is to ensure consistency and fairness of marking amongst
the unit markers.
Inter-marker reliability, i.e. consistency, is notoriously low, and internal moderation
seeks to expose marked papers to a second scrutiny so as to determine the consistency
of marking standards between different unit markers.
External moderation is carried out by the external examiner appointed to the program,
and the aim here is also to monitor consistency and standard of marking to ensure that
students are being assessed fairly
SUBJECT ASSESSMENT PANELS
Some higher education institutions allow board of examiners to delegate
responsibility for the assessment of groups of units to subject assessment panels. The chair of
such panels is normally the head of the school that houses the majority of units, and
membership consists of internal examiners, i.e. staff who taught the units under
consideration, and external examiners.
CHAIRING THE BOARD OF EXAMINERS
The board of examiners is normally chaired by the head of department or his or her
nominee, and the approach should be formal and rigorous. Each meeting should be
numbered, for example ‘fifth meeting of the board of examiners’; this facilitates review of
decisions taken at previous meetings.
The chair should never take the minutes of the meeting, as this would distract him or
her from conducting the meeting in an appropriate manner. The minutes should be headed
‘confidential’, and recorded by an experienced administrator who should identify the chair
and secretary by name.
VALIDATION BY PANEL
The validation panel is comprised of key stakeholders in early childhood education.
The panel plays an important role. The Validity of an instrument refers to the degree to which
an instrument measures what it is supposed to be measuring. For example, a temperature
measuring instrument is supposed to measure only temperature; it cannot be considered a
valid instrument if it measures an attribute other than temperature; it cannot be considered as
valid instrument.
According to TREECE and TREECE, Validity refers to an instrument or test actually
testing what it supposed to be testing
The number and choice of panelists will depend on circumstances and resources.
The need for confidentiality must be stressed. There will be a panel chairperson, who
might also act as a co-coordinator for distribution of the materials well in advance.
The other panelists are chosen for their expertness, the variety of viewpoints they can
contribute, and their number should include some representation from those who will
eventually use the results.
Gender balance should be maintained where possible.
Each item writer whose work is being reviewed acts as secretary to the panel while
those items are being discussed. The item writer using the panel meeting records,
written or taped, to edit or vet the draft items as meticulously and slowly as possible.
The panel will have engaged in a variety of discussions and made many suggestions:
they will have offered hunches about the validity or difficulty of items, have given
their perceptions about the plausibility of distracters, and have pointed out actual
errors of fact or language use.
It is now up to the item writer to respond and accommodate to as many of these
comments as seem sensible from a professional viewpoint.
SUMMARY:
So for we are discussed about question bank-preparation, validation, moderation by
panel, utilisation of introduction, definition, principles, purpose of question bank, planning a
question bank, preparation of question bank, preparing question cards, development of
question bank, process of item analysis, filling and storage of question, review and removal
of unwanted question, computerized question bank, test, purpose of test, types of tests,
selecting item types, editing and validating items, assembling and administering a test,
arrange items, arrangement of questions, dimension of question bank, levels of question bank,
writing question directions, reproducing the test, advantage, disadvantages, grading,
drawbacks of traditional grading system, assigning grades, absolute grades, the grading
system, relative scale and administering the role of faculty
CONCLUSION:
Teachers have to play multifaceted roles as part of their job requirements. They can
contribute significantly in setting a question paper, evaluating the answer script and awarding
the mark/grade of healthy environment and social order.
BIBLIOGRAPHY:
1) B.T BASAVANTHAPPA;“Nursing education”; 1st edition 2004; EMMESS medical
publishers
2) KP NEERAJA; “a textbook of communication and education; 2nd edition; 2006.
3) J. J GULIBERT; “educational handbook for health personnels”; 6th edition; CBC
publishers.
4) MARILYN E.PARKER; “a textbook of nursing education”; 1st edition 2003; JAYPEE
BROTHERS publishers.
5) MR. ELAKUVANA BHASKARA RAJ; “textbook of nursing education”; EMMESS
medical publishers.
6) JASPREET KAUR SODI; textbook of nursing education, 1st edition 2017; JAYPEE
BROTHERS publication.
JOURNAL
1) THE CREATION OF “QUESTIONS BANK” AND INTRODUCTION OF 2.0.
EXAMINATION SESSION ANDRZEJ FILIP, PIOTR DRAG
Author: ANDRZEJ FILIP, PIOTR DRAG
Journal: Information system in management
Distance Learning Centre, Jagiellonian University
The Institute of American Studies and Polish Diaspora, Jagiellonian University
In the Institute of American Studies and Polish Diaspora Jagiellonian University, with the
support of the Distance Learning Centre at the Jagiellonian University was introduced an
innovative method of examination based on empowering the students. During the 2014
session, the students were invited to create test questions. Accepted by the lecturer questions
were used on the exam. Extensive “Questions Bank” may be used in subsequent
examinations 2.0. The authors of the paper present practical advice on how to prepare and
carry out such an examination. They share know how of practical suggestions from
pedagogical to technical aspect of moving from teaching to learning while using the idea of
Questions Bank. They discuss the impact on the motivation and creativity of students, the
principles of achievement and assessment, methods of verifying the content of the questions
and technical measures to make questions and hindering cheating. The use of innovative
methods of preparing and conducting the exam based on the Questions Bank had a positive
impact on the mobilization and involvement of students, which resulted in a very good
performance evaluation questionnaires of the lecturer.
2) AUTOMATIC GENERATION OF QUESTION BANK BASED ON PRE-DEFINED
TEMPLATES
Author: AHMED EZZ AWAD, MOHAMED EHIA DAHAB
Journal: International journal of innovation and advancement in computer science
The preparation of question bank is a difficult and time consuming. This paper
explains an algorithm that provides solution to automatic generation of question bank based
on a set of pre-defined templates. All possible questions are generated by parameterized
concepts from a set of pre-defined templates. The generated questions cover all selected
topics in all level of difficulties the form of a multiple-choice question (MCQ). The outputs
can be used for both examinations and asynchronous training. If the output used in
asynchronous training, it augmented with a set of brief explanation. A successful automatic
generation of question bank has been developed for a biology course (Bio110) in KING
ABDULAZIZ UNIVERSITY (KAU).
3) VALIDATION OF A QUESTION BANK AS PREPARATION FOR THE
EMERGENCY MEDICINE IN- TRAINING EXAMINATION
Author: JOMASELLI P, GOVERNATORI N.
Journal: Western journal of emergency medicine
These results suggest that a questions bank may be useful for predicting performance
on in-training exam scores, Major limitations of the study include small sample size and the
use of one particular question bank. Further research is necessary to compare different study
preparation materials