On Framing the Research Question and Choosing the Appropriate Research Design

Patrick S. Parfrey and Pietro Ravani

Abstract
Clinical epidemiology is the science of human disease investigation with a focus on
diagnosis, prognosis, and treatment. The generation of a reasonable question requires defi
nition of patients, interventions, controls, and outcomes. The goal of research design is to
minimize error, to ensure adequate samples, to measure input and output variables
appropriately, to consider external and internal validities, to limit bias, and to address clinical
as well as statistical relevance. The hierarchy of evidence for clinical decision-making places
randomized controlled trials (RCT) or systematic review of good quality RCTs at the top of
the evidence pyramid. Prognostic and etiologic questions are best addressed with longitudinal
cohort studies.

Introduction
Clinical epidemiology is the science of human disease investigation, with a focus on
problems of most interest to patients: diagnosis, prognosis, and management. Articles are
included in this book on the design and analysis of cohort studies and randomized controlled
trials. Methodological issues involved with studies of biomarkers, quality of life, genetic
diseases, and qualitative research are evaluated. In addition, chapters are presented on the
methodology associated with aggregation of multiple studies such as meta-analysis,
pharmacoeconomics, health technology assessment, and clinical practice guidelines. Finally,
important issues involved in the practice of clinical epidemiology such as ethical approval
and management of studies are discussed. In this chapter we consider how to frame the
research question, error defi nition, measurement, sampling, the choice of research design,
and the difference between clinical relevance and statistical signifi - cance. In addition, here
we provide an overview of principles and concepts that are discussed in more detail in
subsequent chapters.

Framing the Clinical Research Question

The research process usually starts with a general idea or initial problem. Research ideas may
originate from practical clinical problems, request for proposals by funding agencies or
private companies, reading the literature and thinking of ways to extend or refi ne previous
research, or the translation of basic science discoveries to the clinic or the community.
Literature review is always required to identify related research, to defi ne the knowledge
gap, to avoid redundancy when the answer is already clear, and to set the research within a
proper conceptual and theoretical context based on what is already known. The next step is to
generate a researchable question from the general idea. This stage of conceptualization
should generate testable hypotheses and delineate the exposure–outcome relationship to be
studied in a defi ned patient population, whether it be a study of prognosis, diagnosis, or
treatment. Thus operationalization of the proposed study requires characterization of the
specifi c disease to be studied, establishment of the input variable or exposure (test, risk
factor, or intervention) to be associated with an output or clinical outcome. The latter may be
the gold standard in a diagnostic test, the clinical event in a cohort study evaluating risk, or
the primary clinical outcome in a randomized trial of an intervention. Thus, the broad initial
idea is translated into a feasible research project. Narrowing down the area of research is
necessary to formulate an answerable question, in which the target population of the study is
determined along with a meaningful effect measure—the prespecifi ed study outcome. In
framing researchable questions, it is crucial to defi ne the Patients, Interventions, Controls,
and Outcomes (PICO) of relevance. The study question should defi ne the patients (P) to be
studied (e.g., prevalent or incident), through clearly defi ned eligibility criteria. These criteria
should specify the problem, the comorbid conditions to include (because the answer(s) to the
research question may vary by the condition, e.g., diabetes, cardiovascular disease); and those
not to include (because for them the question may be of less interest or hardly answerable,
e.g., those with short expected survival). Secondly, the type of exposure (intervention or
prognostic factor or test; I) is defi ned, and its specifi cs (e.g., what does the exposure actually
comprise). Next, the comparison group (C) is defi ned. Finally, the outcome of interest (O) is
declared. Following consideration of the PICO issues the researchable question can then be
posed, e.g., “Does a particular statin prevent cardiac events, when compared to conventional
therapy, in diabetic patients with stage 3 and 4 chronic kidney disease”? The
operationalization of the study must be consistent with its purpose. If the question is one of
effi cacy (Does it work in the ideal world?), then the measurement tools identifi ed should be
very accurate, may be complex and expensive, and may not be necessarily useful in practice.
Opposite considerations are involved in effectiveness studies (Does it work in the real
world?), and trade-offs between rigor and practicality are necessary. Further operational steps
in clinical research involve limiting error, whether it be random or systematic error,
identifying a representative sample to study, determining a clinically relevant effect to assess,
ensuring that the study is feasible, cost-effective, and ethical.

Error
The goal of clinical research is to estimate population characteristics (parameters)
such as risks by making measurements on samples from the target population. The hope is
that the study estimates be as close as possible to the true values in the population (accuracy)
with little uncertainty (imprecision) around them (Table 1). However, an error component
exists in any study. This is the difference between

TABEL 1

the value observed in the sample and the true value of the phenomenon of interest in the
parent population. There are two main types of error: random or accidental error, and
systematic error ( bias). Random errors are due to chance and compensate since their average
effect is zero. Systematic errors are non-compensating distortions in measurement (Fig. 1).
Mistakes caused by carelessness, or human fallibility (e.g., incorrect use of an instrument,
error in recording or in calculations), may contribute to both random and systematic error.
Both errors arise from many sources and both can be minimized using different strategies.
However, their control can be costly and complete elimination may be impossible. Systematic
error, as opposed to random error, is not limited by increasing the study size and replicates if
the study is repeated. Confounding is a special error since it is due to chance in experimental
designs but it is a bias in non-experimental studies. Confounding occurs when the effect of
the exposure is mixed with that of another variable (confounder) related to both exposure and
outcome, which does not lie in the causal pathway between them. For example if high serum
phosphate levels are found to be associated with higher mortality, it is important to consider
the confounding effect of low glomerular fi ltration rate in the assessment of the relationship
between serum phosphate and death [ 1].
For any given study, the design should aim to limit error. In some cases, pilot studies
are helpful in identifying the main potential sources of error (known sources of variability
and bias— Table 1) such that the design of the main study can control them [ 2, 3]. Some
errors are specifi c to some designs and are discussed in a subsequent chapter of this series.
Both random and systematic errors can occur during all stages of a study, from
conceptualization of the idea to sampling (participant selection) and actual measurements
(information collection).

Sampling
Once the target population has been defi ned, the next challenge is to recruit study
participants representative of the target population. The sampling process is important, as
usually a small fraction of the target population is studied for reasons of cost and feasibility.
Errors in the sampling process can affect both the actual estimate and its precision (Table 1,
Fig. 2). To reduce sampling errors researchers must set up a proper sampling system and
estimate an adequate sample size.
Recruitment of a random sample of the target population is necessary to ensure
generalizability of study results. For example if

Gambar 1. Unbiased sampel and Biased sampel

we wish to estimate the prevalence of Chronic Kidney Disease (CKD) in the general
population, the best approach would be to use random sampling, possibly over-sampling
some subgroup of particular interest (e.g., members of a racial group) in order to have suffi
ciently precise estimates for that subgroup [ 4, 5]. In this instance, a sample of subjects drawn
from a nephrology or a diabetic clinic, any hospital department, school, workplace or people
walking down the street would not be representative of the general population. The likelihood
of CKD may be positively or negatively related to factors associated with receiving care or
working in a particular setting. On the other hand, if a study aimed at understanding the
characteristics of patients with CKD referred to a nephrologist, a study of consecutive
patients referred for CKD would probably provide a reasonably generalizable result.
 If the purpose of the study is to estimate a measure of effect due to some intervention,
then the sampling problem is not finished. Here the comparability of study groups, other than
with regard to the exposure of interest, must be ensured. Indeed to measure the effect of a
therapy, we need to contrast the experience of people given the therapy to those not so
treated. However, people differ from one another in myriad of ways, some of which might
affect the outcome of interest. To avoid such concerns in studies of therapy, random
assignment of study participants to therapy is recommended to ensure comparability of study
groups in the long run. These must be of suffi cient size to reduce the possibility that some
measurable or unmeasurable prognostic factors be associated with one or other of the groups
(random confounding).
The randomization process consists of three interrelated maneuvers: the generation of
random allocation sequences; strategies to promote allocation concealment; and intention-to-
treat analysis. Random sequences are usually generated by means of computer programs. The
use of calendar or treatment days, birth dates, etc. is not appropriate since it does not
guarantee unpredictability. Allocation concealment is meant to prevent those recruiting trial
subjects from the knowledge of upcoming assignment and protect selection biases. Useful
ways to implement concealed allocation include the use central randomization, or the use of
sequentially numbered sealed opaque envelopes. Intention-to-treat analysis consists in
keeping all randomized patients in their original assigned groups during analysis regardless of
adherence or any protocol deviations. This is necessary to maintain group comparability.

Sample Size Estimation

When planning a comparative study two possible random errors (called type I and II errors)
are considered. A type I error is made if the results of a study have a statistically signifi cant
result when in fact there is no difference between study groups. This risk of false negative
results is commonly set at 5 % (equivalent to a signifi cant P value of 0.05). A Type II error is
made if the results of a study are non-signifi cant when in fact a difference truly exists. This
risk of false positive results is usually set at 10 or 20 %. The other factors that determine how
large a study should be are the size of the effect to be detected and the expected outcome
variability. Different formulae exist to estimate the sample size depending on the type of
response variable and the analytical tool used to assess the input– output relationship [ 6]. In
all studies the sample size will depend on the expected variability in the data, effect size
(delta), level of signifi cance (alpha error), and study power (1-beta error).