Annals of the Rheumatic Diseases 1992; 51: 177-181

177

Validity of single variables and composite indices for

measuring disease activity in rheumatoid arthritis
Desiree M F M van der Heijde, Martin A van't Hof, Piet L C M van Riel,
Miek A van Leeuwen, Martin H van Rijswijk, Levinus B A van de Putte

Department of
Rheumatology,
University Hospital
Nijmegen,
The Netherlands
D M F M van der Heiide
P L C M van Riel
L B A van de Putte
Department of Medical
Statistics,
Nijmegen University,
The Netherlands
M A van't Hof
Department of
Rheumatology,
University Hospital
Groningen,
The Netherlands
M A van Leeuwen
M H van Rijswijk
Correspondence

to:

Dr D M F M van der Heilde,

Department of Rheumatology,

University Hospital Nijmegen,

Geert Grooteplein Zuid 8,
6525 GA Nijmegen,
The Netherlands.
Accepted for publication
18 March 1991

Abstract
are used most in measuring the disease process.
There is no agreement as to which variable Some composite indices have also been debest mirrors disease activity in rheumatoid veloped, such as the Mallya and Riel indices and
arthritis (RA) and no studies have been the disease activity score."'0 In contrast to the
performed on the validity of disease activity questionnaires, little research has been carried
variables. In this study the validity of 10 out on the validity of the various variables and
commonly used single variables and three indices. The validity of a variable means that
composite indices was tested.
the variable is measuring what it is supposed to
All patients participated in a large follow up measure. For a process variable in rheumatoid
study in two clinics. The patients (n=233) had arthritis, this variable must measure 'disease
classical or definite RA and a disease duration activity'. This disease activity must be related to
of less than one year at entry. The mean the ultimate result of the disease processs.
follow up time was 30 months; the follow up
A problem in the study of the validity of
frequency was once every four weeks; 6011 process variables in rheumatoid arthritis is that
records were used in the analysis. The valida- a standard for disease activity is not available.
tion criteria included correlations with the Analogous to the validation of questionnaires on
other variables (correlational validity), with physical disability,4 5 we have chosen an explorthe physical disability (criterion validity I), ative method, resulting in a combination of
and with the radiographically determined several aspects of validity. This method for
damage of hands and feet (construct validity). validation is widely used in the social and
The judgment of a group of rheumatologists behavioural sciences and has been recommended
in clinical practice was also used as a model of by Tugwell and Bombardier for the selection of
criterion validity (II).
endpoint measures in clinical trials. " The
In this comparison the disease activity overall results of all these facets of validity give
score and Mallya index showed the best the ultimate conclusion on the validity of a
validity. The best single variable was the particular process variable.
number of swollen joints. The validity of most
Much research has been carried out on
single variables was poor and these variables disease activity variables and on judgment
were not suitable as single endpoint measures analysis,'2 13 although usually only one aspect
in clinical trials.
of disease activity has been evaluated. For
example, the possible effect of a new assessment
Rheumatoid arthritis (RA) is a heterogeneous was checked by the correlation with another
disease with a greatly varying clinical course and disease activity variable such as the Ritchie
outcome. Two aspects are important in evaluatscore or the acute phase response. 14 15 However,
ing the disease: the disease activity and the the reliability of these 'reference' disease activity
ultimate outcome. The assessments to evaluate variables is not well enough known; little
these features are often called 'process' and research has been carried out on the comparison
'outcome' measurements, respectively.' Radio- of different laboratory and clinical disease
graphic abnormalities and physical disability are activity variables.'6 17 As far as we know, no
two tools used to evaluate outcome and much
study has compared many laboratory and clinical
research has been carried out on these methods. disease activity variables and tested their ability
Methods to score radiographic damage have to evaluate the ultimate outcome, measured by
been widely tested-for example, for inter- and radiographically detected damage and physical
intraobserver variations.2 3 Questionnaires to disability in a large database of patient records.
measure physical disability have been developed
The aim of this study was to compare the
and extensively tested and these are valid validity of 10 single variables and three comassessments for measuring outcome.4 5
posite indices. The disease activity variables
Process measurements are widely used in the were chosen arbitrarily from the large number
evaluation of short term clinical trials. Although of assessments available. We selected especially
rheumatologists have discussed in workshops those measures which are frequently used to
which endpoint measures should be used in measure disease activity in clinical trials and in
clinical trials, no agreement exists,6 7 and in clinical practice. The most valid variable to
most clinical trials many variables are used for
measure disease activity in clinical trials and
evaluation. This may cause conflicting results clinical practice was determined.
and, in addition, a correction for multiple
testing is needed. This requires a larger group Patients and methods
of patients to ensure enough power for the PATIENTS
study.. Single clinical and laboratory variables All patients took part in an ongoing prospective

van der Heijde, van't Hof, van Riel, van Leeuwen, van Riiswijk, van de Putte

178

study of RA. Consecutive patients who met the

following criteria were included in the study:
classical or definite RA according to the
American Rheumatism Association criteria,
disease duration shorter than one year at entry
to the study, and not previously treated with
slow acting antirheumatic drugs (SAARDs).
The project is a cooperation between the departments of rheumatology of the University
Hospital Nijmegen and the University Hospital
Groningen in The Netherlands. At time of
analysis 133 patients participated in the study in
Nijmegen (clinic I). The follow up ranged from
8 to 57 months (mean 28; number of check ups
2937). In Groningen (clinic II), 100 patients
participated, with a follow up from 8 to 58
months (mean 31; number of check ups 3074).

composed of the Ritchie index, number of

swollen joints, ESR, and general health.9 A total
of 13 variables were analysed in this study.
Every six months the patients of clinic I
completed a questionnaire on physical disability
comparable with the health assessment questionnaire.5 19 Plain anterior radiographs of the
hands and feet were also made every six
months. The number of erosions and the joint
space narrowing in the hands were scored by a
modified version of Sharp's method.20 The foot
radiographs were scored in a similar way. These
scoring methods have been extensively described
before.2' Summation of the erosions and joint
space narrowing gives the total score. The
scores of all 77 patients of clinic I who were
followed for at least two years were used in the
analysis of radiographic features.

ASSESSMENTS

In the two clinics three specially trained research

nurses assessed all the patients in the outpatient
department. The following assessments were
carried out every four weeks: number of tender
joints and number of swollen joints. All the
following joints were examined bilaterally for
the presence or absence of swelling and tenderness: temporomandibular, sternoclavicular,
acromioclavicular, shoulder, elbow, wrist, each
metacarpophalangeal, each proximal interphalangeal and interphalangeal of the thumb,
hip, knee, ankle, subtalar, midtarsal, metatarsophalangeal, and interphalangeal of the big
toe. The Ritchie articular index,18 time of
morning stiffness (minutes), pain (on a visual
analogue scale of 10 cm, O=no pain, 10=worst
pain possible), general health (visual analogue
scale of 10 cm, O=best possible, 10=worst
possible), grip strength with a vigorimeter
(kPa), erythrocyte sedimentation rate (ESR)
according to Westergren (millimetres in one
hour), haemoglobin (mmol/l), and C reactive
protein (mg/l). The normal range for haemoglobin in our clinics is 1 mmol/l lower for
women than for men. Therefore we decreased
the haemoglobin value of the men by 1 mmol/l,
ensuring comparability regardless of sex.
We calculated the Mallya index of disease
activity for each patient at each visit.8 This
index consists of morning stiffness, pain (visual
analogue scale), grip strength, articular index,
haemoglobin (uncorrected for sex) and ESR.
Each variable is divided into four classes and the
mean score of the six variables gives the full
score (range one to four). The Riel index (a
modified Mallya index) is made up of the
variables morning stiffness, number of tender
joints, haemoglobin (corrected for sex) and ESR
and is calculated in a analogous manner to the
Mallya index.'0 The disease activity score is
Table I Characteristics of the patients at the start of the follow up study

No (%) women
Mean (SD) age (years)
Mean (SD) disease duration (months)
IgM rheumatoid factor >5 IU (No (%))

Physical disability
Erosion of hands and feet (number)
Total score, hands and feet

Clinic I
(n= 133)

Clinic II
(n= 100)

88 (66)
54-3 (14 5)
6-1 (4 0)
103 (80)
0 64 (0 57)
2-9 (4-7)
5-3 (4-7)

65 (65)
47-7 (15-6)
6-9 (4-6)
88 (88)

VALIDATION

As there is no standard available for the

description of the quality of the variables
considered for disease activity, a more explorative validation procedure was chosen. The
following aspects of validation were considered.
Correlational validity: does the measure correlate with other measures which are supposed to
measure disease activity? For this analysis
mutual correlations of the variables under
investigation have been calculated. Criterion
validity: does the assessment fit with the theory
about the disease? Does the method measure
the true clinical status? Therefore, correlations
with physical disability have been determined
and the clinical judgment of rheumatologists on
disease activity has also been analysed. Construct
validity: does the process variable (disease
activity) lead to the ultimate result, the outcome?
Correlations between the disease activity variables and radiographically determined damage
have been calculated.
STATISTICAL PROCEDURES

Initially, all variables were assessed on their

suitability for standard statistical analysis. If
necessary, a transformation to 'normality' was
performed. The transformed values were used
in further analysis. The study of validity was
mainly based on correlations.
Results
Table 1 presents the initial characteristics of the
patients; table 2 summarises the variables used
in the analysis. Owing to a high skewness, some
of the variables are transformed to approximate
a normal distribution. The skewness for clinic
II is comparable with that of clinic I, and
therefore the same transformations were performed.
CORRELATIONAL VALIDITY

The mutual correlations of the different disease

activity variables give the correlational validity.
The first calculations are performed on the data
from each patient: the correlations between the
variables within a patient and thereafter the

Validity of measures ofdisease activity in RA

179

Table 2 Mean, standard deviation (SD), minimum, maximum, and skewness before and after transformation of the variables of clinic I patients (n= 133)
and clinkc II patients (n=100)
Variable
Clinic I (2937 check ups)
Clinic II (3074 check ups)
Mean

Disease activity score

Mallya index
Riel index
General health
Pain*
Tender joints'
Swollen joints
Ritchie index*
Morning stiffnesst

3 35
2-23
1-97
30-7
31-6
10-2
12-1
8-3
40
29
28
7-9
40

ESRt

C reactive proteint
Haemoglobin
Grip strength'

SD

1-23
0-54
0-62
22-6
23-0
8-9
7-3
75
67
24
39
1-0
22

Minimum

Maximum

0-32
1-00
1-00
0
0
0
0
0
0
1
0-1
4-7
0

Skewness
before
transformation

7-26
3-83
4 00
100
100
44
37
46
360
140
260
10-8
138

Skewness
-

3-28
2-18

0-53
0-59
0-78
0-51
1 1
2-78
1-42
2-46
-0-41
0-90

Mean

SD

after
transformation

-0-38
-0-31
-0-11
-

-0-06
0-16
-0-43
-0-61
-

0-07

1-95

28-5
12-9
7-5
11-3
36
31
24
8-0
47

1-28
0-58
0-64

24-9
12-3
7-1

10-9

69
26
36
1-0
26

Minimum

0-35
1-00

1-00

0
0
0
0
0
1
0-1
4-6
0

Maximum
8-35
4-00
4 00

100
48
40
52
360
140
398
11-0
184

*Root transformation.

tDouble root transformation.

tLog transformation; ESR=erythrocyte sedimentation rate.

mean over all patients are calculated, leading to

a mean correlation matrix. The correlation
matrix is the basis on which the mean correlations with the other 12 variables are then
calculated. Table 3 shows that the results of the
correlational validity of the individual patient
data are fully comparable in the two clinics. The
mean correlations of the three composite indices
are comparable and substantially higher than
the single variables. The variable haemoglobin
has an especially low correlational validity
(0-29). The calculations are also performed on
the raw data. Firstly, the correlation of each
variable with every other variable is determined. Thereafter, for every variable we calculated the mean (minimum, maximum) correlation with the other variables. Table 3 shows
that the overall correlations are similar to those
on the individual patient data.
CRITERION VALIDITY

Criterion validity implies that a good disease

activity variable measures the true clinical status
of the patient. Two theoretical aspects are
distinguished.
The first aspect of the criterion validity of all
variables is determined by calculating the
Pearson correlations between the variables and
the physical disability on the individual patient
data. This shows how the disease activity
variables reveal what is the most important
Table 3 Correlational validity: mean Pearson correlation of each variable with the other 12
variables on the raw and individual data of all patients in the two clinics
Variable

Mean individual correlations

Clink II (n=100)

Clinic I (n=2937)

Mean

Range

Mean

Mean

0-63
0-65
0-61
0-38
0-42
0-50
0-43
0-50
0-42
0-47
0-41
0-32
0-41

0-35-0-89
0-51-0-83
0-39-0-83
0-14-0-73
0-18-0-73
0-18-0-92
0-24-0-81
0-19-0-92
0-18-0-75
0-28-0-70
0-23-0-70
0-14-0-59
0-32-0-61

0-64
0-64
0-63

Clinic I (n=133)
Disease activity score
Mallya index
Riel index
General health
Pain
Tender joints
Swollen joints
Ritchie index
Morning stiffness
ESR*
C reactive protein

Raw correlations

Haemoglobin
Grip strength
*ESR=erythrocyte sedimentation rate.

0-42
0-50

0:41

0-51
0-35
0-38
0-40
0-30
0-46

Range
0-34-0-87

0-43-0-87
0-51-0-87
0-06-0-70
0-13-0-93
0-21-0-70
0-11-0-93
0-02-0-61
0-10-0-74
0-13-0-74
0-02-0-58

0-28-0-71

0-61
0-62
0-59
0-36
0-38
0-47
0-38
0-48
0-33
0-36
0-38
0-29
0-41

Range
0-32-0-87

0-44-0-86

0-40-0-86
0-06-0-80
0-03-0-80

0109-094
0-18-0-72
0-08-0-94
0-07-0-64
0-09-0-70
0-09-0-70
0-03-0-59
0-23-0-64

Table 4 Criterion validity I: the mean and median

individual Pearson correlation of each variable with the
physical disability (clinic I, n=133)
Variable

Mean

Disease activity score

0-44
Mallya index
0-43
Riel index
0-33
General health
0-36
Pain
0-39
Tender joints
0-37
Swollen joints
0-36
Ritchie index
0-42
Morning stiffness
0-29
ESR*
0-34
C reactive protein
0-38
Haemoglobin
-0-16
Grip strength
-0-35
*ESR=erythrocyte sedimentation rate.

Median
0-70
0-60
0-50
0-49
0-58
0-58
0-57
0-68
0-47
0-45
0-60
-0-32
-0-58

aspect for the patients. Table 4 gives the mean

and the median correlations. The medians can
be interpreted best. The disease activity score
and the Ritchie index have the highest median
correlations (0-70 and 0-68, respectively); haemoglobin shows the lowest correlation (0-32).
Clinical judgment by rheumatologists is
another procedure to evaluate actual disease
activity. We therefore used the decisions of
rheumatologists in clinical practice to start or
stop treatment with SAARDs. Decisions to
begin treatment with SAARDs or to stop
SAARDs because of a lack of efficacy were
equated with times of high disease activity;
withdrawal as a result of remission, or not
starting or changing SAARDs for at least one
year as times of low disease activity. The disease
activity score is the formalisation of the decisions
of rheumatologists on the start and withdrawal
of treatment with SAARDs in clinical practice
and was constructed for the group of rheumatologists in clinic I. Therefore, only the records
of the patients of clinic II were used for this
analysis. Patients were divided into two groups,
one with high and one with low disease activity,
according to the above explicit rules.9 The
group with high disease activity consists of 129
-records and the group with low disease activity
115 records.
For all variables the mean and standard
deviations of the groups with high and low
disease activity are determined. The larger the
difference between the means of the two groups,

180

van der Heijde, van't Hof, van Riel, van Leeuwen, van Rijswijk, van de Putte

the larger the discriminating power of the

variable. To compare the variables, the differences are standardised (i.e. the difference is
divided by the pooled standard deviation).
Table 5 shows that the disease activity score has
the most discriminating power (SD score 1-66),
followed by the Riel index (SD score 1 46) and
the Mallya index (SD score 1-37). The swollen
joints count is the only single variable with a
considerable discriminating power (SD score

1[35).

Table S Criterion validity II: mean and standard deviation (SD) for high and low disease
activity (DA) in clinic II patients (n=100) and standardised difference
Variable

Criterion validity II

Standardised
difference*

Low DA

High DA
Mean
(n=129)

Disease activity score

4-27
Mallya index
2-62
Riel index
2-41
Pain
6-14
Tender joints
4-01
Swollen joints
12-60
Ritchie index
3-82
Morning stiffness
2-21
ESRt
3 52
C reactive protein
2-27
Haemoglobin
7-44
Grip strength
7-18
*Standardised difference: (mean of high
and low DA group.
tESR=erythrocyte sedimentation rate.

SD

Mean
(n= 115)

SD

1 11
0 58
0-62
2-30
1-63
7 25
1-56
1-31
0-88
0 80
1-07
1-78

2 50
1-89
1-60
3-34
2-19
4-16
2 05
1-06
2-72
1-59
7-88
5-48

101
048
0-47
2-79
1-76
4-93
1 51
1-14
0 93
0-62
0-69
2-32

1-66
1-37
1-46
1 10
1-08
1-35
1 15
0-93
0 89
0 94
049
0-82

DA group-mean of low DA group)/pooled SD of high

Table 6 Construct validity: correlations between increase in joint damage in periods of six
months and two years and mean clinical and laboratoty variables over the same period
(clinic I, n=77)
Variable

Six monthly periods*

Two year periods*

(n= 133)

0-33
0 30
030
0-07
0-12
0-18
0-42
0-17
0 15
0-29
0 35
-0-24
-0-29

0-28
0 30
0-25
0-17
0-19
0-13
0-27
0-14
0 07
0-38
0 43
-0-19
-0-28

0 33
0 34
0-32
0-13
0-17
0-16
0-39
0-17
0-14
0 39
0-46
-0 25
-0-32

0-31
0-25
0-22
0-12
0-18
015
0-54
0-13
0 10
0-19
0 40
-0 10
-0-32

0-26
0 30
0-21
0-23
0-26
0-06
0 39
007
0 03
0-36
0-52
-0-14
-0 39

0 30
0-31
0-24
0-20
0-26
0-12
0-48
0-11
0 10
0-29
0 50
-0-14
-0-38

(n=345)

Disease activity score

Mallya index
Riel index
General health
Pain
Tender joints
Swollen joints
Ritchie index
Morning stiffness

ESRt

C reactive protein
Haemoglobin
Grip strength

*E=erosions, N=joint space narrowing, T=total

tESR=erythrocyte sedimentation

joint

score.

rate.

Table 7 Summary of validation criteria for the 13 variables

Correlational

validity
Disease activity score
Mallya index

Riel index
General health
Pain
Tender joints
Swollen joints
Ritchie index
Morning stiffness
ESR*
C reactive protein
Haemoglobin
Grip strength

++
++
++

Criterion

Criterion

Construct

validity I

validity II

validity

++
+

++
++
++

+
+
+

+
+
+
+

+
+

++

++

+
+

+
+

+
+

+
+
-

++
+

++=very good; +=good; =moderate; -=poor.

*ESR=erythrocyte sedimentation rate.

++
+

CONSTRUCT VALIDITY

Construct validity evaluates whether the

variable which measures the process corresponds
with the ultimate result of the process: the
outcome. Longstanding disease activity leads to
(largely) irreversible joint damage, visible on
radiographs as erosion and joint space narrowing. Thus the damage caused by the disease
process and visible on radiographs is cumulative.
For this analysis only those patients of clinic I
who have been followed for at least two years
have been included (n=77). The mean of each
variable is calculated over adjacent periods of
six months. This mean is correlated with the
increase in joint damage in the same period. In
total, 345 of those periods were available for
analysis. The mean of the variables and the
increase of the joint damage over a period of two
years were also calculated. The total of the two
year periods in the analysis is 133. Table 6 summarises the results of the Spearman rank
correlations of the mean of the variables with
the increase in number of erosions, joint space
narrowing and total score. The results of the six
monthly periods and the two year periods are
comparable. The C reactive protein, swollen
joints, ESR, disease activity score, Mallya
index, Riel index, and grip strength have the
highest correlations. The other variables have
substantially lower correlations.
Discussion
Surprisingly, no systematic studies on the comparison of the validity of 'process' measurements have been made, although these variables
are widely used in clinical trials and in clinical
practice. In this study we were able to evaluate
various aspects of the validity of 13 variables
frequendy used to measure disease activity. Not
only single variables were included, but also
indexes such as the Mallya, Riel, and disease
activity score. Owing to the lack of a standard
for disease activity we used an explorative
method to evaluate validity, as is used widely in
the social sciences. Physical disability and radiographically detected damage in the hands and
feet are mostly used as 'outcome' measurements.Y5 Variables which measure the 'process'
(disease activity) have to correspond with the
'outcome'.' Therefore we used the physical
disability and the radiographically detected
damage in the hands and feet to validate the
disease activity variables.
Table 7 summarises all types of validity and
gives a review of the relative quality of the
variables. The limits of '+' are determined
arbitrarily. However, this was carried out to
make an overall comparison of all aspects of the
validity of all variables more easily. This table
provides a practical guide for the choice of a
process variable. The disease activity indices
(especially the disease activity score and the
Mallya index) score best overall. The validity
of various single variables was very poor
(especially haemoglobin, morning stiffness,
general health, pain, and grip strength). The
most valid single variable is the number of
swollen joints. The relation between swollen
joints and radiographic progression confirms

Validity of measures of disease activity in RA

earlier reports."2 The disease activity score is

composed of the Ritchie index, the number of
swollen joints, the ESR, and general health.9
The Mallya index consists of morning stiffness,
pain, grip strength, articular index, haemoglobin, and ESR.8 Thus the combination of a
few variables, which have little value as a single
measurement, into an index greatly enhances
the validity.
The disease activity score is based on judgments in clinical practice. This disease activity
score turned out to be one of the most valid
variables to measure disease activity compared
with widely used variables. Therefore it might
be concluded that clinical judgment by rheumatologists correlates with the ultimate outcome, namely physical disability and radiographically detectable damage.
In clinical trials of SAARDs, many disease
activity variables are used to evaluate the effect.
This introduces the problem of multiple testing,
which may lead to conflicting results in several
variables. In addition, more patients are needed
to diminish a false positive result (type I error).
This study shows that many of the single
variables have a (very) low validity in measuring
disease activity and that the disease activity
score and Mallya index are the most valid
variables in measuring disease activity. Therefore these indices may be used as single endpoint measures in clinical trials, preventing
conflicting results and multiple testing (and
fewer patients are needed). Not only in trials,
but also in patient management, the problem
arises of the discordance of the courses of
individual variables of disease activity. In this
situation, the indices may also be helpful.
In conclusion, we recommend using a disease
activity index in evaluating trials and in clinical
practice. Most of the so called disease activity
variables only partly measure disease activity. It
is questionable whether these variables, such as
haemoglobin and morning stiffness, should be
used as single disease activity variables. However, in combination with others variables in a
disease activity index, these variables may be
useful.
This investigation was supported by a grant from the Program for
the Stimulation of Health Research (SGO) and the Netherlands
League against Rheumatism. We thank H Theunisse, D Lubberts,
and W Lolkema, research nurses, for the collection of the clinical
data and Dr N Houtman for referral of patients.
1

Fries J F. Toward an understanding of patient outcome

measurement. Arthritis Rheum 1983; 26: 697-704.

181
2 Sharp J T, Bluhm G B, Brook A, et al. Reproducibility of
multiple-observer scoring of radiologic abnormalities in the
hands and wrists of patients with rheumatoid arthritis.
Arthritis Rheum 1985; 28: 16-24.
3 Sharp J T. Radiologic assessment as an outcome measure in
rheumatoid arthritis. Arthritis Rheum 1989; 32: 221-9.
4 Meenan R F, Gertman P M, Mason J H. The arthritis impact
measurement scales. Arthritis Rheum 1982; 25: 1048-53.
5 Fries J F, Spitz P, Kraines R G, Holman H R. Measurement
of patient outcome in arthritis. Arthritis Rheum 1980; 23:
137-45.
6 Bombardier C, Tugwell P, Sinclair A, Dok C, Anderson G,
Buchanan W. Preference for endpoint measures in clinical
trials: results of structured workshops. J Rheumatol 1982;
9: 798-801.
7 Scott D L, Spector T D, Pullar T, McConkey B. What should
we hope to achieve when treating rheumatoid arthritis? Ann
Rheum Dis 1989; 48: 256-61.
8 Mallya R K, Mace B E W. The assessment of disease activity
in rheumatoid arthritis using a multivariate analysis.
Rheumatol Rehabil 1981; 20: 14-7.
9 van der Heijde D M F M, van't Hof M A, van Riel P L C M,
van Leeuwen M A, van Rijswijk M H, van de Putte L B A.
Judging disease activity in clinical practice in rheumatoid
arthritis. The first step in the development of a 'disease
activity score'. Ann Rheum Dis 1990; 49: 916-20.
10 van Riel P L C M, van de Putte L B A, Gribnau F W J,
Macrae K D. Comparison of auranofin and aurothioglucose
in the treatment of rheumatoid arthritis: a single-blind
study. Clin Rheumatol 1984; 3 (suppl 1): 51-6.
11 Tugwell P, Bombardier C. A methodologic framework for
developing and selecting endpoints in clinical trials. J
Rheumatol 1982; 9: 758-62.
12 Kirwan J R, Chaput de Saintonge D M, Joyce C R B, Currey
H L F. Clinical judgment in rheumatoid arthritis I.
Rheumatologists' opinions and the development of paper
patients. Ann Rheum Dis 1983; 42: 644-7.
13 Kirwan J R, Chaput de Saintonge D M, Joyce C R B, Holmes
J, Currey H L F. Inability of rheumatologists to describe
their true policies for assessing rheumatoid arthritis. Ann
Rheum Dis 1986; 45: 156-61.
14 March R E, Kirwan J R, Reeback J S, Holborow E J. IgM,
IgG and IgA rheumatoid factors (antiglobulins) in early
rheumatoid arthritis and their production of articular index
over one year. Scand J Rheumatol 1987; 16: 407-11.
15 Thompson P W, Silman A J, Kirwan J R, Currey H L F.
Articular indices of joint inflammation in rheumatoid
arthritis. Arthritis Rheum 1987; 30: 618-23.
16 Bull B S, Levy W C, Westengard J C, et al. Ranking of
laboratory tests by consensus analysis. Lancet 1986; ii:
377-80.
17 Bull B S, Farr M, Meyer P J, Westengard J C, Bacon P A,
Stuart J. Efficacy of tests used to monitor rheumatoid
arthritis. Lancet 1989; ii: 965-7.
18 Ritchie D M, Boyle J A, McInnes J M, et al. Clinical studies
with an articular index for the assessments of joint
tenderness in patients with rheumatoid arthritis. Q J Med
1968; 147: 393-406.
19 van der Heiide D M F M, van Riel P L C M, van de Putte L B
A. Sensitivity of a Dutch health assessment questionnaire
in a trial comparing hydroxychloroquine vs. sulphasalazine.
Scand J Rheumatol 1990; 19: 407-12.
20 Sharp J T, Young D Y, Bluhm G B, et al. How many joints in
the hands and wrists should be included in a score of
radiologic abnormalities used to assess rheumatoid arthritis.
Arthritis Rhewn 1985; 28: 1326-35.
21 van der Heiide D M, van Riel P L, Nuver-Zwart H H,
Gribnau F W, van de Putte L B. Effects of hydroxychloroquine and sulphasalazine on progression of joint damage in
rheumatoid arthritis. Lancet 1989; i: 1036-8.
22 Sharp J T, Lidsky M D, Duffy J. Clinical responses during
gold therapy for rheumatoid arthritis. Changes in synovitis,
radiologically detectable erosive lesions, serum proteins,
and serologic abnormalities. Arthritis Rhewn 1982; 25:
540-9.
23 Ingeman-Nielsen M, Haslkov 0, Hansen T M, Halberg P,
Stage P, Lorenzen I. Clinical synovitis and radiological
lesions in rheumatoid arthritis. A prospective study of 25
patients during treatment with remission-inducing drugs.
ScandJ Rheunatol 1983; 12: 237-40.