Research Method for MPH/MSc.

Merga Dheresa (PhD, MBA, MPH, Bsc.)

1
 VARIABLES
‘What information are we going to collect in our
study to meet our objectives?’

2
FORMULATING VARIABLES
What is a variable?
• A VARIABLE is a characteristic of a person, object or phenomenon which can take
on different values.
• In the form of numbers (e.g., age) or
• non-numerical characteristics (e.g., sex)
• A simple example of a variable in the form of numbers is ‘a person’s age’.
• Other examples of variables are:
• weight (expressed in kilograms or in pounds);
•home - clinic distance (expressed in kilometres or in minutes walking distance);
• monthly income (expressed in dollars, rupees, or kwachas); and
• number of children (1, 2, etc.).
3
• What is a variable?
• Because the values of all these variables are expressed in numbers, we call them
NUMERICAL VARIABLES.
• Some variables may also be expressed in categories.
• For example, the variable sex has two districts categories, groups, male and female.

4
5
Numerical variables can either be continuous or discrete
1. Continuous
With this type of data, one can develop more and more accurate measurements
depending on the instrument used, e.g.:
• height in centimeters (2.5 cm or 2.546 cm or 2.543216 cm)
• temperature in degrees Celsius (37.20 C or 37.199990C etc.)
2. Discrete
These are variables in which numbers can only have full values, e.g.:
• number of visits to a clinic (0, 1, 2, 3, 4, etc).
• number of sexual partners (0, 1, 2, 3, 4, 5, etc.)

6
Categorical variables, on the other hand, can either be ordinal or nominal
1. Ordinal variables These are grouped variables that are ordered or ranked in
• Increasing or Decreasing order:
• High income (above 300 per month)
• Middle income (100-300 per month)
• Low income (less than 100 per month)
Other examples are:
• Disability:
• No disability,
• Partial disability,
• Serious or total disability

7
Categorical variables, on the other hand, can either be ordinal or nominal
1. Ordinal variables
• Seriousness of a disease:
• severe,
• moderate,
• mild
• Agreement with a statement:
• fully agree,
• partially agree,
• fully disagree
• Fear of leprosy:
• will not share food with a patient;
• will not enter the house of a patient;
• will not allow patient to live in the community.
8
Categorical variables, on the other hand, can either be ordinal or nominal

2. Nominal variables.
The groups in these variables do not have an order or ranking in them.
For example:
Sex: male, female
Main food crops: maize, millet, rice, etc.
Religion: Christian, Muslims, Hindu, Buddhism, etc.

9
Variables
• When you selected the variables for your study, you did so with the
assumption that they either would help to define your problem
(dependent variables) and its different components or that they were
contributory factors to your problem (independent variables)

• The purpose of data analysis is to identify whether these assumptions

were correct or not, and to highlight possible new views on the
problem under study

• The ultimate purpose of analysis is to answer the research questions

outlined in the objectives with your data.
10
Operationalizing variables by choosing appropriate indicators

• Note that the different values of many of the variables presented up to

now can easily be determined
• However, for some variables it is sometimes not possible to find
meaningful categories unless the variables are made operational with
one or more precise INDICATORS

• Operationalizing variables means that you make them ‘measurable’

11
Operationalizing variables by choosing appropriate indicators

• If you want to determine the level of knowledge concerning a specific issue in order
to find out to what extent the factor ‘poor knowledge’

• The variable ‘level of knowledge’ cannot be measured as such.

• You would need to develop a series of questions, for example on pre-natal care and
risk factors related to pregnancy.
• The answers to these questions form an indicator of someone’s knowledge on this
issue, which can then be categorized
• If 10 questions were asked, you might decide that the knowledge of those with:
- 0 to 3 correct answers is poor,
- 4 to 6 correct answers is reasonable, and
- 7 to 10 correct answers is good.
12
Operationalizing variables by choosing appropriate indicators

• Nutritional status of under-5 year olds: widely used indicators for nutritional
status include:
- Weight in relation to age (W/A)
- Weight in relation to height (W/H)
- Height in relation to age (H/A)
- Upper-arm circumference (UAC)
• For the classification of nutritional status, internationally accepted categories
already exist, which are based on so-called standard growth curves.
• For the indicator ‘Weight/Age’, for example, children are:
- well-nourished if they are above 80% of the standard,
- moderately malnourished if they are between 60% and 80%,
- severely malnourished if they are below 60%. 13
Operationalizing variables by choosing appropriate indicators

• When defining variables on the basis of the problem analysis diagram,

it is important to realize which variables are measurable as such and
which ones need indicators

• Once appropriate indicators have been identified we know exactly

what information we are looking for
• This makes the collection of data as well as the analysis more focused
and efficient

14
Defining variables and indicators of variables
• To ensure that everyone understands exactly what has been measured
and to ensure that there will be consistency in the measurement, it is
necessary to clearly define the variables (and indicators of variables)

• For example, to define the indicator ‘waiting time’ it is necessary to

decide what will be considered the starting point of the ‘waiting
period’ e.g., is it when the patient enters the front door, or when he
has been registered and obtained his card?

15
Dependent and independent variables
• Because in health systems research you often look for causal
explanations, it is important to make a distinction between dependent
and independent variables

• The variable that is used to describe or measure the problem under

study is called the DEPENDENT (outcome) variable

• The variables that are used to describe or measure the factors that are
assumed to cause or at least to influence the problem are called the
INDEPENDENT variables

16
Dependent and independent variables
• In a study of the relationship between smoking and lung cancer, ‘suffering
from lung cancer’ (with the values yes, no) would be the dependent variable
and ‘smoking’ is the independent variable

• Whether a variable is dependent or independent is determined by the

statement of the problem and the objectives of the study.
• It is therefore important when designing an analytical study to clearly state
which variable is the dependent and which are the independent ones
• Note that if a researcher investigates why people smoke, ‘smoking’ is the
dependent variable, and ‘pressure from peers to smoke’ could be an
independent variable.
• In the lung cancer study ‘ smoking’ was the independent variable.
17
Background variables
• In almost every study, BACKGROUND VARIABLES, such as age,
sex, educational level, socioeconomic status, marital status and
religion, should be considered
• These background variables are often related to a number of
independent variables, so that they influence the problem indirectly
(hence they are called background variables)

• Background variables are notorious ‘confounders’.

18
Questionnaire designs and
data sources

19
 Questionnaire designs and data sources

• Questionnaire design
• Types of questions
• Steps in Designing a questionnaire
• Consideration to write Open or Closed
• Formatting interpreting questionnaires
• Data sources
• Data collection
• Data quality control

20
 Questionnaire design

• The quality of research depends to a large extent on the quality of the

data collection tools

• Designing good ‘questioning tools’ forms an important and time-

consuming phase in the development of most research proposals

• Questionnaires are an inexpensive way to gather data from a potentially

large number of respondents

21
 Questionnaire …

• Points to be considered before designing questionnaire

1. What exactly do we want to know (i.e. the objectives)

2. Is Questionnaire the right technique to obtain all answers

3. Who will be the respondents

4. What techniques will we use

5. How large is the sample that will be interviewed

22
 Types of questions

1. Open-ended questions: permit free response

• Important to explore in-depth information on
1. Issues with which the researcher is not familiar
2. Opinion, attitudes, or sensitive questions

2. Closed questions: supplies the respondent with two or more specified alternative
responses

23
 Steps in designing a questionnaire

1. Content : take objectives and variables as a starting point

2. Formulating questions: formulate one or more questions that will provide the information
needed for each variable
3. Sequencing the questions: Design your interview or questionnaire to be ‘informant
friendly’.
4. Formatting the questionnaire
5. Translation
6. Check Reliability (Pre‐ test)

24
 Writing the Questionnaire

• Points to be Considered to write open or closed format and interpreting

questionnaires
1. Clarity: questions must be clear and unambiguous
2. Leading Questions: A leading question is one that forces or implies a certain
type of answer
3. Phrasing of questions
4. Embarrassing Questions
5. Hypothetical Questions, eg If I were a him ……

25
 Types of Data

• Data may be classified as

1. Cross-sectional data: are collected at one point in time
2. Time series, or panel data: are collected on repeated occasions

26
 Types of data…
• Based on Type of Research Methods: data can be classified into

1. Quantitative data : is that which can be easily measured and recorded in numerical
form

2. Qualitative data : is information that is represented by means other than numbers

27
 Data sources

• Major Data Sources of Population

– Census
– Vital statistics
– Surveys
– Medical/administrative records
• Data may be collected from
– Primary source eg. Interview
– Secondary sources eg. Records

28
 Data collection

Types of data collection

• Interviewing
• Self‐ administered questionnaire
• Observation
• Focus group discussion
• Using secondary data
• Routine records
• Database from previous studies etc…

29
 Factors that affect data collection methods

• Resource (human and financial)

• Ethical issues
• Sensitivity of the information gathered
• Geographic accessibility
• Time
• Language
• Study design
• Study participants

30
 Field staff recruitment

• Select data collectors in a way to minimize bias

• Every one who will be participating in the field work should be trained
• Pre‐test is usually done as part of the training of field staffs
– At the end of the training data collectors and supervisors go to the field to
test the data collection instrument

– Pre‐test is usually done on similar population but in an area different from the
actual survey

31
 Purpose of pre‐test

• Check the clarity of instruments

• Assess the level of understanding of data collectors and supervisors
• Do data collectors understand the questions and thus administer properly
• Correction of the instruments before the actual study
• To estimate the time and budget needed for the actual data collection

32
 Quality control at field level

• Ensure appropriate administration of the data collection instrument

• Monitoring and supervision of field activities:
– Supervision should be done during data collection
– Intensive supervision is especially needed at the beginning of data collection

33
 Monitoring at the field

• Adherence to the study protocol

• Consistency of protocol implementation

• Appropriate implementation of ethical issues

• Completeness of questionnaires

34
 Quality control at data entry and processing

• Questionnaires should be manually edited before entering data into computer

software
• Proper design of the data entry template
• Double entry to check consistency
• Preparing Data dictionary is mandatory
• Data cleaning
• Always save the master database on a separate file before doing any coding and
categorizing

35
 Quality control for Qualitative studies

• Triangulation of data using different methods

• Proper labelling and documenting of data base

– Labelling tapes
– Interview notes
– Observations

36
 Data processing

• It refers to data entry into a computer, and data checks and

correction

• The purpose of this process is to produce a relatively clean data

set

37
 Coding Data

• Computers are at their best with numbers

• Some statistical packages cannot analyse alphabetical codes

• Code: a numerical and/or alphabetical system for classifying

information
• Coding: translation of information: E.g. questionnaire
responses, into numbered categories for data entry in a data
processing system 38
 Coding data …

• Example, instead of using male and female for the variable sex, it can be coded as
1=male and 2=female

• The meaning of the codes will depend on the level of measurement of the variable

• Nominal: codes are just indications of the category

• Ordinal: codes are indications of ordering

• Interval/Ratio: codes are the actual numerical value

39
 Coding data …

• Before entering data into a computer for processing develop a

code that indicates how each variable is coded (whether it is
actual or coded values) and in what field it appears

• Assign a unique ID for each study subject or questionnaire

• The ID number can direct the person back to the original data for
correction
• Used for follow-up
40
 Data entry

• It concerns the transfer of data from questionnaire into a

computer file
• Converting data in to that can be read and manipulated by
computers used in quantitative data analysis

• Data entry may be possible to begin anytime anywhere

• ‘GIGO’ Garbage In, Garbage Out

41
 Data Cleaning

• Data cleaning/editing is the process of identifying values in your data

that are unusual or unexpected and examining them to decide if the data
is correct or if there is an error

• Before we analyse data we need to clean errors may have occurred in

data collection, coding, or data entry
• The purpose of this process is to produce a clean set of data for statistical
analysis

42
Measurement Error

43
Measurement
• Measurements vary from questions asked about:
• Symptoms during history-taking,
• Physical examinations and
• Tests,
• laboratory tests,
• imaging techniques,
• self-report questionnaires,

44
Characteristics of measurements

• From diagnosis to outcome measurements

• From clinician-based to patient-based measurements
• From objective to subjective measurements
• From unidimensional to multidimensional characteristics
• From observable to non-observable characteristics

45
Development of a measurement instrument

46
Measurement Error
• Measurement error is the difference between the recorded response to a question
and the ‘true’ value.
• Measurement error occurs as part of data collection,
• It may arise from four sources:

1. The questionnaire,

2. The data collection method,

3. The interviewer, and

4. The respondent
47
Sources of Measurement Error
1. Questionnaire Effects
1.1. Specification problems:
• Error can occur because the data specification is inadequate and/or
inconsistent with what the survey requires

• Specification problems can occur due to poorly worded questionnaires

and survey instructions, or may occur due to the difficulty of
measuring the desired concept
• These problems exist because of inadequate specifications of uses and
needs, concepts, and individual data elements

48
1. Questionnaire Effects
1.2. Question wording:
• The designer wants the respondent to interpret the question as the designer
would interpret the question
The potentials for error are many
• First, the questionnaire designer may not have a clear formulation of the
concept he/she is trying to measure
• Next, even if he/she has a clear concept, it may not be clearly represented in the
question.
• Even if the concept is clear and faithfully reproduced, the respondent may not
interpret the request as intended.
• Not all respondents will understand the request for information, due to
language or cultural differences, affective response to the wording, or
differences in experience and context between the questionnaire author and the
respondent

49
1. Questionnaire Effects
1.3. Length of the questions
• The questionnaire designer is faced with the dilemma of keeping
questions short and simple while assuring sufficient information is
provided to respondents so they are able to answer a question
accurately and completely.
• Common sense and good writing practice tell us that keeping
questions short and simple will lead to clear interpretation.
• Research, however, suggests that longer questions actually yield more
accurate detail from respondents than shorter questions, at least as they
relate to behavioral reports of symptoms and doctors’ visits

50
1. Questionnaire Effects

1.4. Length of the questionnaire

• Long questionnaires may introduce error due to respondent fatigue or
loss of concentration.
• Length of the questionnaire may also be related to non-response error.
• There is always a tension between the desire to ask as many questions as
possible and the awareness that error may be introduced if the
questionnaire is too long

51
1. Questionnaire Effects
1.5. Open and closed formats
• Question formats in which respondents are asked to respond using a
specified set of options (closed format) may yield different responses
than when respondents are not given categories (open format).
• A given response is less likely to be volunteered by a respondent in an
open format than when included as an option in a closed format.
• The closed format may remind respondents of something they may not
have otherwise remembered to include.
• The response options to a question cue the respondent as to the level
or type of responses considered appropriate

52
1. Questionnaire Effects
1.6. Data Collection Mode Effects
Face-to-face interviewing
• Face-to-face interviewing is the mode in which an interviewer administers a structured
questionnaire to respondents.
• Using a paper questionnaire or via computer assisted personal interviewing, the
interviewer completes the questionnaire by asking questions of the respondent.
• One problem for face-to-face interviewing is the effect of interviewers on respondents’
answers to questions, resulting in increases to the variances of survey estimates.
• Another possible source of measurement error is the presence of other household
members who may affect the respondent’s answers. T
• his is especially true for topics viewed as sensitive by the respondents.
• Measurement error may also occur because respondents are reluctant to report
socially undesirable traits or acts

53
1. Questionnaire Effects

1.7. Skipping pattern

• while use skipping pattern the interviewer and/or interviewee may use
the advantage of the skipping and follow the pattern that shorten the
time of the question by choosing the response that let them jump to
other question.
1.8. Reference period:
• The length the reference period at which the problems occurs may
affect the response of the respondents.
• If it is so long the respondents may not exactly recall the problem
(telescoping bias )

54
 2. Respondent Effects
• Survey respondents vary considerably in their abilities and willingness to
provide accurate answers to questions regarding to their behaviors.
• Respondent behaviors can be understood within the framework of the
generally accepted cognitive model of survey response, which recognizes
four basic tasks required from respondents when they are answering each
survey question.
a) question interpretation,
(b) memory retrieval,
(c) judgment formation,
(d) response editing
• This is a useful model for understanding how variability across respondents
may influence the quality of self-reported information

55
 2. Respondent Effects
2.1. Question Interpretation
• Respondents sometimes employ terminology that differs from that employed in
research questionnaires.
• A related concern is the degree to which respondent cultural background may
influence the interpretation and/or comprehension of survey questions.
• Some disease patterns and risk practices are known to vary cross-culturally and those
varied experiences and beliefs regarding the problem can also be expected to influence
respondent knowledge and familiarity with the topic in general and related
terminology in particular.
• Experienced researchers, of course, recognize the importance of investigating and
addressing these potential problems by employing focus groups discussion

56
2. Respondent Effects

2.2. Memory Retrieval

• The accuracy of respondent recall has been the focus of much
attention.
• Poorly worded survey questions may present respondents with
difficult cognitive challenges in terms of the effort necessary to
retrospectively retrieve specific and/or detailed information that may
not be readily accessible in memory

57
2. Respondent Effects

2.3. Response Editing

• Once respondents have successfully interpreted a survey question and
retrieved the relevant information necessary to form an answer, they must
decide whether that answer is to be accurately shared with the researcher.
• Given the illicit and sometimes stigmatizing nature of behaviors,
conventional wisdom often suggests that some respondents will make
conscious decisions to underreport, or deny altogether, any such
behavior.
• That survey respondents will sometimes attempt to present themselves in a
favourable, albeit not completely accurate, light during survey interviews
is well understood and is commonly referred to as social desirability bias
58
2. Respondent Effects

2.4. Recall bias

• if the presence of disease influences the perception of its causes
(rumination bias) or the search for exposure to the putative cause
(exposure suspicion bias), or in a trial if the patient knows what they
receive may influence their answers (participant expectation bias)

59
2. Respondent Effects

2.5. Reporting bias; participants can ‘‘collaborate’’ with researchers

and give answers in the direction they perceive are of interest
(obsequiousness bias), or the existence of a case triggers family
information (family aggregation bias)

60
 3. Interviewer Effects
• Because of individual differences, each interviewer handles the survey
situation in a different way, that is, in asking questions, probing and
recording answers, or interacting with the respondent, some
interviewers appear to obtain different responses from others.
• The interviewer situation is dynamic and relies on an interviewer
establishing rapport with the respondent.
• Interviewers may not ask questions exactly as worded, follow skip
patterns correctly or probe for answers non directivity.
• They may not follow directions exactly, either purposefully or because
those directions have not been made clear enough.
• Interviewers may vary their inflection, tone of voice, or other personal
mannerisms without even knowing it.
61
3. Interviewer Effects

• Errors, both over reports and underreports, can occur for each interviewer.
• When over reporting and underreporting of approximately the same
magnitude occurs, small interviewer bias will result.
• However, these individual interviewer errors may be large and in the same
direction, resulting in large errors for individual interviewers Another
possible mechanism that may account for interviewer effects involves social
distance.
• It is possible that the social distance between respondents and interviewers
may influence respondent willingness to report sensitive behaviors
• Interviewer-respondent familiarity with one another may also influence the
quality of self-reported behaviours or practice

62
3. Interviewer Effects
• Helping the respondents in different ways (even with gestures), putting emphases
in different questions, misreading questions, failing to probe answers correctly,
not following other elements of standardized survey protocols
• Social distance: if the interviewer varies from interviewee by sex, culture,
education, age, and dressing; the respondents may not feel comfort to genuinely
and freely respond about their morbidity and other contributing factors.
• Interviewer-respondent familiarity with one another may also influence the
quality of self-reported behaviours or practice
• Between interviewer bias: variability in terms of knowledge, profession
experience may influence the result.
• Language barrier; if the interviewers are not fluent speakers of the language of
respondents, he/she cannot explain the question in case the respondents unable to
understand the questionnaire

63
 4. Context Effects
• Various aspects of the social and physical environment within which
survey data are collected may also influence the quality of the information
collected
• One aspect of the social environment that has received attention is the
absence or presence of other individuals during the interview, as this is
believed to influence the social desirability demands or pressures that
respondents may perceive.
• In general, the presence of others during survey interviews is known to be
associated with lower reporting of sensitive behaviors

• The physical context within which interviews take place may also
influence social desirability pressures and self-report quality

64
 5. Processing Errors.
• Once data collection is complete, the construction of a final survey data set
requires the implementation of numerous coding and editing rules.
• The integrity of these rules is particularly critical in surveys, as they
typically involve assumptions about the reporting intentions of respondents
5.1. Data Entry Errors
• Data entry errors occur in the process of transferring collected data to an
electronic medium.
• The frequency of these errors varies by the types of information collected
(e.g., numeric versus character) and the mode of data collection.
• For example, with paper and pencil enumeration, survey data are key-
entered after the survey interview takes place

65
5. Processing Errors.

5.2.Key entry errors

• Double key entry is an effective technique for minimizing key-entry
errors in paper and pencil instruments.
• With this technique, key-entered data are independently keyed a second
time, usually by a different key-entry operator, and the resulting two
data sets are compared.
• Discrepancy reports identifying cases and survey items for which
differences exist are then reviewed, and erroneous data are corrected.
• In some cases only a sample of questionnaires is verified to test the
accuracy of the key-entry process and of individual key-entry personnel

66
5. Processing Errors.
5.3. Pre-Edit Coding Errors
• Most surveys require some type of pre-edit coding of the survey returns before
they can be further processed in edit, imputation, and summary systems.
• The required coding is generally of two types—unit and item response
coding.
• The unit response coding assigns and records the status of the interview.
• It is designed to indicate the response status of the return for a sampled unit so
that it can be appropriately handled in subsequent processing.

67
5. Processing Errors.
5.3. Pre-Edit Coding Errors
• Item response coding is the more commonly discussed form of
questionnaire coding
• This can involve coding an actual response for a survey question into a
category
• This situation occurs for questions that elicit open-ended responses.
• For example, sometimes one of the responses in categorical questions is
“other-specify,” which results in a free-text response.
• In other surveys, the respondent may be asked by design to provide an
open-ended response

68
5. Processing Errors.
5.3. Pre-Edit Coding Errors
• The recoding of open-ended responses into a categorical variable is
performed by coders who interpret and catalogue each response.
• This process can result in error or bias, since different coders are likely to
interpret and code some responses differently.
• Even the same coders may change the way they code as they gain more
experience or get bored. Another problem with open-ended responses is that
respondents sometimes supply more than one answer to the question
• A technique employed to measure and reduce coding errors is the use of
multiple coders for the same set of responses.
• This is analogous to double key entry.
• Once both sets of coders have completed the coding, a comparison is made
between the two sets of data, with any discrepancies resolved by committee or
by an expert 69
5. Processing Errors.
5.4. Editing Errors
• Editing is a procedure designed and used for detecting erroneous and/or
questionable survey data (survey response data or identification type data)
with the goal of correcting (manually and/or via electronic means) as much
erroneous data (not necessarily all of the questioned data) as possible,
usually prior to data imputation and summary procedures
• Editing generally occurs at various points in the survey processing and can,
in itself, generate errors at each juncture.
• The editing process, in general, allows survey managers to review each
report for accuracy an activity that usually results in a feeling of control
over the process while obtaining a sense of the data 

70
5. Processing Errors
5.4. Editing Errors
• While, indeed, there are benefits from editing, recent studies
documented by various survey organizations have shown that data are
often over-edited.
• This over editing unnecessarily uses valuable resources and can
actually add more error to the data than it eliminates.
• Processing error can also arise during edit processing due to edit
model failure in an automated system.
• Whether the editing is based on a very sophisticated mathematical
model or on simple range checks, the manner in which erroneous data
are flagged, and how they are handled, can introduce processing error

71
RELIABILITY &
VALIDITY

72
RELIABILITY
Reliability is defined as ‘the degree to which the measurement is free
from measurement error
• In full this is ‘the extent to which scores for patients who have not
changed are the same for repeated measurement under several
conditions:
• internal consistency
• test–retest
• inter-rater
• intra-rater

73
Types of Reliability
• Parameters of reliability for continuous variables
1. Intraclass correlation coefficients for single measurements
2. Pearson’s r
• Parameters of measurement error for continuous variables
1. Standard error of measurement
2. Coefficient of variation
• Parameters of reliability for categorical variables
1. Cohen’s kappa for nominal variables
• No parameters of measurement error for categorical variables
• It can be examined, however, which percentage of the measurements are
classified in the same categories. We call this the percentage of agreement
• Cronbach’s alpha as a reliability parameter

74
 Validity
Define: ‘the degree to which an instrument truly measures the
construct(s) it purports to measure’
Types of validity
1. Content validity (including face validity): is the degree to which the
content of a measurement instrument is an adequate reflection of the
construct to be measured’
• If the construct we want to measure is body weight, a weighing scale is
sufficient.
• To measure the construct of obesity, defined as a body mass index (BMI =
weight/height2 ) > 30 kg/m2 , a weighing scale and a measuring rod are needed.

75
Types of validity
A. Face validity :A first aspect of content validity is face validity. ‘the
degree to which a measurement instrument, indeed, looks as though
it is an adequate reflection of the construct to be measured’
It is a subjective assessment and, therefore, there are no standards with regard to
how it should be assessed, and it cannot be quantified.
B. Content validity; When an instrument has passed the test of face
validation, we have to consider its content in more detail
• The purpose of a content validation study is to assess whether the
measurement instrument adequately represents the construct under
study.
• We again emphasize the importance of a good description of the
construct to be measured.

76
2. Criterion validity; ‘the degree to which the scores of a measurement instrument
are an adequate reflection of a gold standard’
• This implies that criterion validity can only be assessed when a gold standard
(i.e. a criterion) is available.
3. Construct validity: In situations in which a gold standard is lacking, construct
validation should be used to provide evidence of validity.
• Construct validity was defined as the degree to which the scores of a measurement
instrument are consistent with hypotheses, e.g. with regard to internal
relationships, relationships with scores of other instruments or differences
between relevant groups
• Construct validation is often considered to be less powerful than criterion
validation

77
4. Cross-cultural: is defined as ‘the degree to which the performance of the items
on a translated or culturally adapted PRO instrument are an adequate reflection
of the performance of items in the original version of the instrument’

• This is often assessed after the translation of a questionnaire.

• Apart from differences induced by the translations, there may also be
differences in cultural issues
• Some items in a questionnaire may be irrelevant in other cultures
• Cross-cultural validation starts with an accurate translation process
• After the translation, or cultural adaptation, the real cross-cultural validation
takes place.

78
Validity and reliability

• The four boxes in this figure reflect various

combinations of validity and reliability.
• The dots represent multiple measurements of one
patient, and the cross within the circle represents the
true score:
• The dots in cell A correspond to valid and quite
reliable scores
• The dots in cell B correspond to mostly invalid,
and definitely unreliable scores
• The dots in cell C correspond to invalid, but
quite reliable scores and
• The dots in cell D correspond to invalid and
unreliable scores

79
80
Thank you

81