NOTICE: This standard has either been superseded and replaced by a new version or withdrawn.

Contact ASTM International (www.astm.org) for the latest information

An American National Standard

Designation: D 6300 – 03

Standard Practice for

Determination of Precision and Bias Data for Use in Test
Methods for Petroleum Products and Lubricants1
This standard is issued under the fixed designation D 6300; the number immediately following the designation indicates the year of
original adoption or, in the case of revision, the year of last revision. A number in parentheses indicates the year of last reapproval. A
superscript epsilon (e) indicates an editorial change since the last revision or reapproval.

INTRODUCTION

Both Research Report RR:D02–1007, Manual on Determining Precision Data for ASTM Methods
on Petroleum Products and Lubricants2 and the ISO 4259, benefitted greatly from more than 50 years
of collaboration between ASTM and the Institute of Petroleum (IP) in the UK. The more recent work
was documented by the IP and has become ISO 4259.
ISO 4259 encompasses both the determination of precision and the application of such precision
data. In effect, it combines the type of information in RR:D02–10072 regarding the determination of
the precision estimates and the type of information in Practice D 3244 for the utilization of test data.
The following practice, intended to replace RR:D02–1007,2 differs slightly from related portions of
the ISO standard. This new practice is consistent with the computer software, ADJD6300 D2PP,
Version 4.43, Determination of Precision and Bias Data for Use in Test Methods for Petroleum
Products.

1. Scope D 3244 Practice for Utilization of Test Data to Determine

1.1 This practice covers the necessary preparations and Conformance with Specifications4
planning for the conduct of interlaboratory programs for the E 29 Practice for Using Significant Digits in Test Data to
development of estimates of precision (determinability, repeat- Determine Conformance with Specifications5
ability, and reproducibility) and of bias (absolute and relative), E 456 Terminology Relating to Quality and Statistics5
and further presents the standard phraseology for incorporating E 691 Practice for Conducting an Interlaboratory Study to
such information into standard test methods. Determine the Precision of a Test Method5
1.2 This practice is generally limited to homogeneous prod- 2.2 ISO Standards:
ucts with which serious sampling problems do not normally ISO 4259 Petroleum Products-Determination and Applica-
arise. tion of Precision Data in Relation to Methods of Test6
1.3 This practice may not be suitable for solid or semisolid 2.3 ASTM Adjuncts:
products such as petroleum coke, industrial pitches, paraffin ADJD6300 D2PP, Version 4.43, Determination of Preci-
waxes, greases, or solid lubricants when the heterogeneous sion and Bias Data for Use in Test Methods for Petroleum
properties of the substances create sampling problems. In such Products7
instances, use Practice E 691 or consult a trained statistician. 3. Terminology
1.4 A software program (ADJD6300) performs the neces-
sary computations prescribed by this practice. 3.1 Definitions:
3.1.1 analysis of variance (ANOVA), n—a procedure for
2. Referenced Documents dividing the total variation of a set of data into two or more
2.1 ASTM Standards: parts, one of which estimates the error due to selecting and
D 123 Terminology Relating to Textiles3 testing specimens and the other part(s) possible sources of
added variation. D 123
1
This practice is under the jurisdiction of ASTM Committee D02 on Petroleum
Products and Lubricants and is the direct responsibility of Subcommittee D02.94 on
4
Quality Assurance and Statistics. Annual Book of ASTM Standards, Vol 05.02.
5
Current edition approved July 10, 2003. Published September 2003. Originally Annual Book of ASTM Standards, Vol 14.02.
6
approved in 1998. Last previous edition approved in 2002 as D 6300–02. Available from International Organization for Standardization, 1 rue de
2
Supporting data have been filed at ASTM International Headquarters and may Varembé, Case postale 56, CH-1211 Geneva 20, Switzerland.
7
be obtained by requesting Research Report RR:D02–1007. Available from ASTM International Headquarters. Order Adjunct No.
3
Annual Book of ASTM Standards, Vol 07.01. ADJD6300.

Copyright © ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959, United States.

1
 D 6300 – 03
3.1.2 bias, n—the difference between the population mean 3.1.11 repeatability, n—the quantitative expression of the
of the test results and an accepted reference value. E 456 random error associated with a single operator in a given
3.1.3 bias, relative, n—the difference between the popula- laboratory obtaining repetitive results by applying the same test
tion mean of the test results and an accepted reference value, method with the same apparatus under constant operating
which is the agreed upon value obtained using an accepted conditions on identical test material within a short interval of
reference method for measuring the same property. time on the same day. It is defined as the difference between
3.1.4 degrees of freedom, n—the divisor used in the calcu- two such results at the 95 % confidence level. RR:D02–1007
lation of variance. 3.1.11.1 Discussion—Interpret as the value equal to or
3.1.4.1 Discussion—This definition applies strictly only in below which the absolute difference between two single test
the simplest cases. Complete definitions are beyond the scope results obtained in the above conditions may expect to lie with
of this practice. ISO 4259 a probability of 95 %. ISO 4259
3.1.5 determinability, n—a quantitative measure of the vari- 3.1.11.2 Discussion—The difference is related to the repeat-
ability associated with the same operator in a given laboratory ability standard deviation but it is not the standard deviation or
obtaining successive determined values using the same appa- its estimate. RR:D02–1007
ratus for a series of operations leading to a single result; it is 3.1.12 reproducibility, n—a quantitative expression of the
defined as that difference between two such single determined random error associated with different operators from different
values as would be exceeded in the long run in only one case laboratories using different apparatus, each obtaining a single
in 20 in the normal and correct operation of the test method. result by applying the same test method on an identical test
3.1.5.1 Discussion—This definition implies that two deter- sample. It is defined as the 95 % confidence limit for the
mined values, obtained under determinability conditions, difference between two such single and independent results.
which differ by more than the determinability value should be 3.1.12.1 Discussion—Interpret as the value equal to or
considered suspect. If an operator obtains more than two below which the absolute difference between two single test
determinations, then it would usually be satisfactory to check results on identical material obtained by operators in different
the most discordant determination against the mean of the laboratories, using the standardized test, may be expected to lie
remainder, using determinability as the critical difference (1).8 with a probability of 95 %. ISO 4259
3.1.6 mean square, n— in analysis of variance, a contrac- 3.1.12.2 Discussion—The difference is related to the repro-
tion of the expression “mean of the squared deviations from the ducibility standard deviation but is not the standard deviation
appropriate average(s)” where the divisor of each sum of or its estimate. RR:D02–1007
squares is the appropriate degrees of freedom. D 123
3.1.12.3 Discussion—In those cases where the normal use
3.1.7 normal distribution, n—the distribution that has the
of the test method does not involve sending a sample to a
probability function:
testing laboratory, either because it is an in-line test method or
f~x! 5 ~1/s! ~2p!21/2exp @2 ~x–µ! 2/2s2# (1) because of serious sample instabilities or similar reasons, the
precision test for obtaining reproducibility may allow for the
where: use of apparatus from the participating laboratories at a
x = a random variate,
common site (several common sites, if feasible). The statistical
µ = the mean distribution, and
s = the standard deviation of the distribution. analysis is not affected thereby. However, the interpretation of
the reproducibility value will be affected, and therefore, the
(Syn. Gaussian distribution, law of error) D 123
precision statement shall, in this case, state the conditions to
3.1.8 outlier, n—a result far enough in magnitude from
which the reproducibility value applies.
other results to be considered not a part of the set.
RR:D02–1007 3.1.13 standard deviation, n—the most usual measure of the
3.1.9 precision, n—the degree of agreement between two or dispersion of observed values or results expressed as the
more results on the same property of identical test material. In positive square root of the variance. E 456
this practice, precision statements are framed in terms of 3.1.14 sum of squares, n—in analysis of variance, a con-
repeatability and reproducibility of the test method. traction of the expression “sum of the squared deviations from
3.1.9.1 Discussion—The testing conditions represented by the appropriate average(s)” where the average(s) of interest
repeatability and reproducibility should reflect the normal may be the average(s) of specific subset(s) of data or of the
extremes of variability under which the test is commonly used. entire set of data. D 123
Repeatability conditions are those showing the least variation; 3.1.15 variance, n—a measure of the dispersion of a series
reproducibility, the usual maximum degree of variability. Refer of accepted results about their average. It is equal to the sum of
to the definitions of each of these terms for greater detail. the squares of the deviation of each result from the average,
RR:D02–1007 divided by the number of degrees of freedom.
3.1.10 random error, n—the chance variation encountered RR:D02–1007
in all test work despite the closest control of variables. 3.1.16 variance, between-laboratory, n—that component of
RR:D02–1007 the overall variance due to the difference in the mean values
obtained by different laboratories. ISO 4259
3.1.16.1 Discussion—When results obtained by more than
8
The bold numbers in parentheses refer to a list of references at the end of this one laboratory are compared, the scatter is usually wider than
practice. when the same number of tests are carried out by a single

2
 D 6300 – 03
laboratory, and there is some variation between means obtained operator, the results will be found satisfactory for judging the
by different laboratories. Differences in operator technique, compliance of the material with the specification. Statements
instrumentation, environment, and sample “as received” are addressing precision and bias are required in ASTM test
among the factors that can affect the between laboratory methods. These then give the user an idea of the precision of
variance. There is a corresponding definition for between- the resulting data and its relationship to an accepted reference
operator variance. material or source (if available). Statements addressing deter-
3.1.16.2 Discussion—The term “between-laboratory” is of- minability are sometimes required as part of the test method
ten shortened to “laboratory” when used to qualify represen- procedure in order to provide early warning of a significant
tative parameters of the dispersion of the population of results, degradation of testing quality while processing any series of
for example as “laboratory variance.” samples.
3.2 Definitions of Terms Specific to This Standard: 5.2 Repeatability and reproducibility are defined in the
3.2.1 determination, n—the process of carrying out a series precision section of every Committee D02 test method. Deter-
of operations specified in the test method whereby a single minability is defined above in Section 3. The relationship
value is obtained. among the three measures of precision can be tabulated in
3.2.2 operator, n—a person who carries out a particular test. terms of their different sources of variation (see Table 1).
3.2.3 probability density function, n—function which yields 5.2.1 When used, determinability is a mandatory part of the
the probability that the random variable takes on any one of its Procedure section. It will allow operators to check their
admissible values; here, we are interested only in the normal technique for the sequence of operations specified. It also
probability. ensures that a result based on the set of determined values is
3.2.4 result, n—the final value obtained by following the not subject to excessive variability from that source.
complete set of instructions in the test method. 5.3 A bias statement furnishes guidelines on the relationship
3.2.4.1 Discussion—It may be obtained from a single de- between a set of test results and a related set of accepted
termination or from several determinations, depending on the reference values. When the bias of a test method is known, a
instructions in the method. When rounding off results, the compensating adjustment can be incorporated in the test
procedures described in Practice E 29 shall be used. method.
5.4 This practice is intended for use by D02 subcommittees
4. Summary of Practice in determining precision estimates and bias statements to be
4.1 A draft of the test method is prepared and a pilot used in D02 test methods. Its procedures correspond with ISO
program can be conducted to verify details of the procedure 4259 and are the basis for the Committee D02 computer
and to estimate roughly the precision of the test method. software, Calculation if Precision Data: Petroleum Test Meth-
4.2 A plan is developed for the interlaboratory study using ods. The use of this practice replaces that of Research Report
the number of participating laboratories to determine the RR:D02–1007.2
number of samples needed to provide the necessary degrees of 5.5 Standard practices for the calculation of precision have
freedom. Samples are acquired and distributed. The interlabo- been written by many committees with emphasis on their
ratory study is then conducted on an agreed draft of the test particular product area. One developed by Committee E11 on
method. Statistics is Practice E 691. Practice E 691 and this practice
4.3 The data are summarized and analyzed. Any depen- differ as outlined in Table 2.
dence of precision on the level of test result is removed by
transformation. The resulting data are inspected for uniformity 6. Stages in Planning of an Interlaboratory Test Program
and for outliers. Any missing and rejected data are estimated. for the Determination of the Precision of a Test
The transformation is confirmed. Finally, an analysis of vari- Method
ance is performed, followed by calculation of repeatability, 6.1 The stages in planning an interlaboratory test program
reproducibility, and bias. When it forms a necessary part of the are: preparing a draft method of test (see 6.2), planning and
test procedure, the determinability is also calculated. executing a pilot program with at least two laboratories
(optional but recommended for new test methods) (see 6.3),
5. Significance and Use planning the interlaboratory program (see 6.4), and executing
5.1 ASTM test methods are frequently intended for use in the interlaboratory program (see 6.5). The four stages are
the manufacture, selling, and buying of materials in accordance described in turn.
with specifications and therefore should provide such precision 6.2 Preparing a Draft Method of Test—This shall contain
that when the test is properly performed by a competent all the necessary details for carrying out the test and reporting

TABLE 1 Sources of Variation

Method Apparatus Operator Laboratory Time
Reproducibility Complete Different Different Different Specified
(Result)
Repeatability Complete Same Same Same Almost same
(Result)
Determinability Incomplete Same Same Same Almost same
(Part result)

3
 D 6300 – 03
TABLE 2 Differences in Calculation of Precision in Practices 6.4 Planning the Interlaboratory Program:
D 6300 and E 691
6.4.1 There shall be at least five participating laboratories,
Element This Practice Practice E 691 but it is preferable to exceed this number in order to reduce the
Applicability Limited in general to Permits heterogeneous number of samples required and to make the precision state-
homogeneous samples for samples.
which serious sampling
ment as representative as possible of the qualified user popu-
problems do not normally lation.
arise. 6.4.2 The number of samples shall be sufficient to cover the
Number of duplicates Two Any number range of the property measured, and to give reliability to the
precision estimates. If any variation of precision with level was
Precision is written Test method Each sample observed in the results of the pilot program, then at least five
for
samples shall be used in the interlaboratory program. In any
Outlier tests: Sequential Simultaneous case, it is necessary to obtain at least 30 degrees of freedom in
Within laboratories Cochran test k-value both repeatability and reproducibility. For repeatability, this
Between Hawkins test h-value
laboratories means obtaining a total of at least 30 pairs of results in the
program.
Outliers Rejected, subject to Rejected if many
subcommittee approval. laboratories or for cause
6.4.3 For reproducibility, Fig. 1 gives the minimum number
such as blunder or not of samples required in terms of L, P, and Q, where L is the
following method. number of participating laboratories, and P and Q are the ratios
Retesting not generally Laboratory may retest
of variance component estimates (see 8.3.1) obtained from the
permitted. sample having rejected pilot program. Specifically, P is the ratio of the interaction
data. component to the repeats component, and Q is the ratio of the
Rejection limit 20 % 5%
laboratories component to the repeats component.

Analysis of variance Two-way, applied globally One-way, applied to each

NOTE 1—Appendix X1 gives the derivation of the equation used. If Q
to all the remaining data sample separately. is much larger than P, then 30 degrees of freedom cannot be achieved; the
at once. blank entries in Fig. 1 correspond to this situation or the approach of it
(that is, when more than 20 samples are required). For these cases, there
Precision multiplier t =2 , where t is the two- 2.8=1.96 =2 is likely to be a significant bias between laboratories. The program
tailed Student’s t for 95 % organizer shall be informed; further standardization of the test method
probability.
may be necessary.
Increases with decreasing Constant.
laboratories 3 samples
6.5 Executing the Interlaboratory Program:
particularly below 12. 6.5.1 One person shall oversee the entire program, from the
distribution of the texts and samples to the final appraisal of the
Variation of precision Minimized by data User may assess from
with level transformation. Equations individual sample results. He or she shall be familiar with the test method, but
for repeatability and precisions. should not personally take part in the actual running of the
reproducibility are generated tests.
in the retransformation
process. 6.5.2 The text of the test method shall be distributed to all
the laboratories in time to raise any queries before the tests
begin. If any laboratory wants to practice the test method in
the results. Any condition which could alter the results shall be advance, this shall be done with samples other than those used
specified. The section on precision will be included at this stage in the program.
only as a heading. 6.5.3 The samples shall be accumulated, subdivided, and
6.3 Planning and Executing a Pilot Program with at Least distributed by the organizer, who shall also keep a reserve of
Two Laboratories: each sample for emergencies. It is most important that the
6.3.1 A pilot program is recommended to be used with new individual laboratory portions be homogeneous. Instructions to
test methods for the following reasons: (1) to verify the details each laboratory shall include the following:
in the operation of the test; (2) to find out how well operators 6.5.3.1 The agreed draft method of test;
can follow the instructions of the test method; (3) to check the 6.5.3.2 Material Safety Data Sheets, where applicable, and
precautions regarding sample handling and storage; and (4) to the handling and storage requirements for the samples;
estimate roughly the precision of the test. 6.5.3.3 The order in which the samples are to be tested (a
6.3.2 At least two samples are required, covering the range different random order for each laboratory);
of results to which the test is intended to apply; however, 6.5.3.4 The statement that two test results are to be obtained
include at least 12 laboratory-sample combinations. Test each in the shortest practical period of time on each sample by the
sample twice by each laboratory under repeatability conditions. same operator with the same apparatus. For statistical reasons
If any omissions or inaccuracies in the draft method are it is imperative that the two results are obtained independently
revealed, they shall now be corrected. Analyze the results for of each other, that is, that the second result is not biased by
precision, bias, and determinability (if applicable) using this knowledge of the first. If this is regarded as impossible to
practice. If any are considered to be too large for the technical achieve with the operator concerned, then the pairs of results
application, then consider alterations to the test method. shall be obtained in a blind fashion, but ensuring that they are

4
 D 6300 – 03

FIG. 1 Determination of Number of Samples Required (see 6.4.3)

carried out in a short period of time (preferably the same day). results, and any unusual occurrences. The unit of accuracy for
The term blind fashion means that the operator does not know reporting the results shall be specified. This should be, if
that the sample is a duplicate of any previous run. possible, more digits reported than will be used in the final test
6.5.3.5 The period of time during which repeated results are method, in order to avoid having rounding unduly affect the
to be obtained and the period of time during which all the estimated precision values.
samples are to be tested; 6.5.3.7 When it is required to estimate the determinability,
6.5.3.6 A blank form for reporting the results. For each the report form must include space for each of the determined
sample, there shall be space for the date of testing, the two values as well as the test results.

5
 D 6300 – 03
6.5.3.8 A statement that the test shall be carried out under lines parallel to the m-axis, then no transformation is necessary.
normal conditions, using operators with good experience but If, however, the plotted points describe non-horizontal straight
not exceptional knowledge; and that the duration of the test lines or curves of the form D = f1(m) and d = f2(m), then a
shall be the same as normal. transformation will be necessary.
6.5.4 The pilot program operators may take part in the 7.2.3 The relationships D = f1(m) and d = f2(m) will not in
interlaboratory program. If their extra experience in testing a general be identical. The statistical procedures of this practice
few more samples produces a noticeable effect, it will serve as require, however, that the same transformation be applicable
a warning that the test method is not satisfactory. They shall be both for repeatability and for reproducibility. For this reason
identified in the report of the results so that any such effect may the two relationships are combined into a single dependency
be noted. relationship D = f (m) (where D now includes d) by including
6.5.5 It can not be overemphasized that the statement of a dummy variable T. This will take account of the difference
precision in the test method is to apply to test results obtained between the relationships, if one exists, and will provide a
by running the agreed procedure exactly as written. Therefore, means of testing for this difference (see A4.1).
the test method must not be significantly altered after its 7.2.4 The single relationship D = f(m) is best estimated by
precision statement is written. weighted linear regression analysis. Strictly speaking, an
iteratively weighted regression should be used, but in most
7. Inspection of Interlaboratory Results for Uniformity cases even an unweighted regression will give a satisfactory
and for Outliers approximation. The derivation of weights is described in A4.2,
7.1 Introduction: and the computational procedure for the regression analysis is
7.1.1 This section specifies procedures for examining the described in A4.3. Typical forms of dependence D = f(m) are
results reported in a statistically designed interlaboratory given in A3.1. These are all expressed in terms of at most two
program (see Section 6) to establish: (2) transformation parameters, B and B0.
7.1.1.1 The independence or dependence of precision and 7.2.5 The typical forms of dependence, the transformations
the level of results; they give rise to, and the regressions to be performed in order
7.1.1.2 The uniformity of precision from laboratory to to estimate the transformation parameters B, are all summa-
laboratory, and to detect the presence of outliers. rized in A3.2. This includes statistical tests for the significance
of the regression (that is, is the relationship D = f(m) parallel
NOTE 2—The procedures are described in mathematical terms based on
the notation of Annex A1 and illustrated with reference to the example
to the m-axis), and for the difference between the repeatability
data (calculation of bromine number) set out in Annex A2. Throughout and reproducibility relationships, based at the 5 % significance
this section (and Section 8), the procedures to be used are first specified level. If such a difference is found to exist, or if no suitable
and then illustrated by a worked example using data given in Annex A2. transformation exists, then the alternative methods of Practice
NOTE 3—It is assumed throughout this section that all the deviations are E 691 shall be used. In such an event it will not be possible to
either from a single normal distribution or capable of being transformed test for laboratory bias over all samples (see 7.6) or separately
into such a distribution (see 7.2). Other cases (which are rare) would
estimate the interaction component of variance (see 8.2).
require different treatment that is beyond the scope of this practice. See (2)
for a statistical test of normality. 7.2.6 If it has been shown at the 5 % significance level that
NOTE 4—Although the procedures shown here are in a form suitable for there is a significant regression of the form D = f(m), then the
hand calculation, it is strongly advised that an electronic computer be used appropriate transformation y = F(x), where x is the reported
to store and analyze interlaboratory test results, based on the procedures of result, is given by the equation
this practice. ADJD6300 D2PP, Version 4.43, Determination of Precision
*f~x!
dx
and Bias Data for Use in Test Methods for Petroleum Products, has been F~x! 5 K (2)
designed for this purpose.
7.2 Transformation of Data: where K = a constant. In that event, all results shall be
7.2.1 In many test methods the precision depends on the transformed accordingly and the remainder of the analysis
level of the test result, and thus the variability of the reported carried out in terms of the transformed results. Typical trans-
results is different from sample to sample. The method of formations are given in A3.1.
analysis outlined in this practice requires that this shall not be 7.2.7 The choice of transformation is difficult to make the
so and the position is rectified, if necessary, by a transforma- subject of formalized rules. Qualified statistical assistance may
tion. be required in particular cases. The presence of outliers may
7.2.2 The laboratories’ standard deviations Dj, and the affect judgement as to the type of transformation required, if
repeats standard deviations dj (see Annex A1) are calculated any (see 7.7).
and plotted separately against the sample means mj. If the 7.2.8 Worked Example:
points so plotted may be considered as lying about a pair of 7.2.8.1 Table 3 lists the values of m, D, and d for the eight

TABLE 3 Computed from Bromine Example Showing Dependence of Precision on Level

Sample Number 3 8 1 4 5 6 2 7
m 0.756 1.22 2.15 3.64 10.9 48.2 65.4 114
D 0.0669 (14) 0.159 (9) 0.729 (8) 0.211 (11) 0.291 (9) 1.50 (9) 2.22 (9) 2.93 (9)
d 0.0500 (9) 0.0572 (9) 0.127 (9) 0.116 (9) 0.0943 (9) 0.527 (9) 0.818 (9) 0.935 (9)

6
 D 6300 – 03
samples in the example given in Annex A2, correct to three TABLE 4 Absolute Differences Between Transformed Repeat
significant digits. Corresponding degrees of freedom are in Results: Bromine Example
parentheses. Inspection of the values in Table 3 shows that both Laboratory Sample

D and d increase with m, the rate of increase diminishing as m 1 2 3 4 5 6 7 8

increases. A plot of these figures on log-log paper (that is, a A 42 21 7 13 7 10 8 0
graph of log D and log d against log m) shows that the points B 23 12 12 0 7 9 3 0
C 0 6 0 0 7 8 4 0
may reasonably be considered as lying about two straight lines D 14 6 0 13 0 8 9 32
(see Fig. A4.1 in Annex A4). From the example calculations E 65 4 0 0 14 5 7 28
F 23 20 34 29 20 30 43 0
given in A4.4, the gradients of these lines are shown to be the
G 62 4 78 0 0 16 18 56
same, with an estimated value of 0.638. Bearing in mind the H 44 20 29 44 0 27 4 32
errors in this estimated value, the gradient may for convenience J 0 59 0 40 0 30 26 0
be taken as 2/3.

*x
2 1
–
3 dx 5 3x3 (3)
0.0782
7.2.8.2 Hence, the same transformation is appropriate both 0.0439 5 0.138 (4)
for repeatability and reproducibility, and is given by the
equation. Since the constant multiplier may be ignored, the where 0.138 is the result obtained by electronic calculation of
transformation thus reduces to that of taking the cube roots of unrounded factors in the expression. There are 72 ranges and
the reported bromine numbers. This yields the transformed as, from Table A2.2, the criterion for 80 ranges is 0.1709, this
data shown in Table A1.3, in which the cube roots are quoted ratio is not significant.
correct to three decimal places. 7.3.4 Uniformity of Reproducibility:
7.3 Tests for Outliers: 7.3.4.1 The following outlier tests are concerned with es-
7.3.1 The reported data or, if it has been decided that a tablishing uniformity in the reproducibility estimate, and are
transformation is necessary, the transformed results shall be designed to detect either a discordant pair of results from a
inspected for outliers. These are the values which are so laboratory on a particular sample or a discordant set of results
different from the remainder that it can only be concluded that from a laboratory on all samples. For both purposes, the
they have arisen from some fault in the application of the test Hawkins’ test (4) is appropriate.
method or from testing a wrong sample. Many possible tests 7.3.4.2 This involves forming for each sample, and finally
may be used and the associated significance levels varied, but for the overall laboratory averages (see 7.6), the ratio of the
those that are specified in the following subsections have been largest absolute deviation of laboratory mean from sample (or
found to be appropriate in this practice. These outlier tests all overall) mean to the square root of certain sums of squares
assume a normal distribution of errors. (A1.6).
7.3.2 Uniformity of Repeatability—The first outlier test is 7.3.4.3 The ratio corresponding to the largest absolute
concerned with detecting a discordant result in a pair of repeat deviation shall be compared with the critical 1 % values given
results. This test (3) involves calculating the eij2 over all the in Table A1.5, where n is the number of laboratory/sample cells
laboratory/sample combinations. Cochran’s criterion at the 1 % in the sample (or the number of overall laboratory means)
significance level is then used to test the ratio of the largest of concerned and where v is the degrees of freedom for the sum
these values over their sum (see A1.5). If its value exceeds the of squares which is additional to that corresponding to the
value given in Table A2.2, corresponding to one degree of sample in question. In the test for laboratory/sample cells v will
freedom, n being the number of pairs available for comparison, refer to other samples, but will be zero in the test for overall
then the member of the pair farthest from the sample mean laboratory averages.
shall be rejected and the process repeated, reducing n by 1, 7.3.4.4 If a significant value is encountered for individual
until no more rejections are called for. In certain cases, samples the corresponding extreme values shall be omitted and
specifically when the number of digits used in reporting results the process repeated. If any extreme values are found in the
leads to a large number of repeat ties, this test can lead to an laboratory totals, then all the results from that laboratory shall
unacceptably large proportion of rejections, for example, more be rejected.
than 10 %. If this is so, this rejection test shall be abandoned 7.3.4.5 If the test leads to an unacceptably large proportion
and some or all of the rejected results shall be retained. A of rejections, for example, more than 10 %, then this rejection
decision based on judgement will be necessary in this case. test shall be abandoned and some or all of the rejected results
7.3.3 Worked Example— In the case of the example given in shall be retained. A decision based on judgement will be
Annex A2, the absolute differences (ranges) between trans- necessary in this case.
formed repeat results, that is, of the pairs of numbers in Table 7.3.5 Worked Example:
A1.3, in units of the third decimal place, are shown in Table 4. 7.3.5.1 The application of Hawkins’ test to cell means
The largest range is 0.078 for Laboratory G on Sample 3. The within samples is shown below.
sum of squares of all the ranges is 7.3.5.2 The first step is to calculate the deviations of cell
0.0422 + 0.0212 + . . . + 0.0262 + 02 = 0.0439. means from respective sample means over the whole array.
Thus, the ratio to be compared with Cochran’s criterion is These are shown in Table 5, in units of the third decimal place.

7
 D 6300 – 03
TABLE 5 Deviations of Cell Means from Respective Sample largest of the corresponding sums of squares (laboratories or
Means: Transformed Bromine Example repeats, as appropriate) to their total (see A1.5). If the ratio
Sample exceeds the critical value given in Table A2.2, with n as the
Laboratory 1 2 3 4 5 6 7 8 number of samples and v the degrees of freedom, then all the
A 20 8 14 15 10 48 6 3 results from the sample in question shall be rejected. In such an
B 75 7 20 9 10 47 6 3 event care should be taken that the extreme standard deviation
C 64 35 3 20 30 4 22 25
D 314 33 18 42 7 39 80 50
is not due to the application of an inappropriate transformation
E 32 32 30 9 7 18 18 39 (see 7.1), or undetected outliers.
F 75 97 31 20 30 8 74 53 7.4.4 There is no optimal test when standard deviations are
G 10 34 32 20 20 61 9 62
H 42 13 4 42 13 21 8 50
based on different degrees of freedom. However, the ratio of
J 1 28 22 29 14 8 10 53 the largest variance to that pooled from the remaining samples
Sum of Squares 117 15 4 6 3 11 13 17 follows an F-distribution with v1 and v2 degrees of freedom
(see A1.7). Here v1 is the degrees of freedom of the variance in
question and v2 is the degrees of freedom from the remaining
The sum of squares of the deviations are then calculated for samples. If the ratio is greater than the critical value given in
each sample. These are also shown in Table 5 in units of the A2.6, corresponding to a significance level of 0.01/S where S is
third decimal place. the number of samples, then results from the sample in
7.3.5.3 The cell to be tested is the one with the most extreme question shall be rejected.
deviation. This was obtained by Laboratory D from Sample 1. 7.4.5 Worked Example:
The appropriate Hawkins’ test ratio is therefore: 7.4.5.1 The standard deviations of the transformed results,
after the rejection of the pair of results by Laboratory D on
0.314
B* 5 5 0.7281 (5) Sample 1, are given in Table 6 in ascending order of sample
=0.117 1 0.015 1 . . . 1 0.017 mean, correct to three significant digits. Corresponding degrees
7.3.5.4 The critical value, corresponding to n = 9 cells in of freedom are in parentheses.
sample 1 and v = 56 extra degrees of freedom from the other 7.4.5.2 Inspection shows that there is no outlying sample
samples is interpolated from Table A1.5 as 0.3729. The test among these. It will be noted that the standard deviations are
value is greater than the critical value, and so the results from now independent of the sample means, which was the purpose
Laboratory D on Sample 1 are rejected. of transforming the results.
7.3.5.5 As there has been a rejection, the mean value, 7.4.5.3 The values in Table 7, taken from a test program on
deviations, and sum of squares are recalculated for Sample 1, bromine numbers over 100, will illustrate the case of a sample
and the procedure is repeated. The next cell to be tested will be rejection.
that obtained by Laboratory F from Sample 2. The Hawkins’ 7.4.5.4 It is clear, by inspection, that the laboratories stan-
test ratio for this cell is: dard deviation of Sample 93 at 15.76 is far greater than the
0.097 others. It is noted that the repeats standard deviation in this
B* 5 5 0.3542 (6) sample is correspondingly large.
=0.006 1 0.015 1 . . . 1 0.017
7.4.5.5 Since laboratory degrees of freedom are not the
7.3.5.6 The critical value corresponding to n = 9 cells in same over all samples, the variance ratio test is used. The
Sample 2 and v = 55 extra degrees of freedom is interpolated variance pooled from all samples, excluding Sample 93, is the
from Table A1.5 as 0.3756. As the test ratio is less than the sum of the sums of squares divided by the total degrees of
critical value there will be no further rejections. freedom, that is
7.4 Rejection of Complete Data from a Sample:
7.4.1 The laboratories standard deviation and repeats stan- ~8 3 5.102 1 9 3 4.202 1 ... 1 83 3.852!
5 19.96 (7)
dard deviation shall be examined for any outlying samples. If ~8 1 9 1 ... 1 8!
a transformation has been carried out or any rejection made, 7.4.5.6 The variance ratio is then calculated as
new standard deviations shall be calculated.
15.262
7.4.2 If the standard deviation for any sample is excessively
19.96 5 11.66 (8)
large, it shall be examined with a view to rejecting the results
from that sample. where 11.66 is the result obtained by electronic calculation
7.4.3 Cochran’s criterion at the 1 % level can be used when without rounding the factors in the expression.
the standard deviations are based on the same number of 7.4.5.7 From Table A1.8 the critical value corresponding to
degrees of freedom. This involves calculating the ratio of the a significance level of 0.01/8 = 0.00125, on 8 and 63 degrees

TABLE 6 Standard Deviations of Transformed Results: Bromine Example

Sample number 3 8 1 4 5 6 2 7
m 0.9100 1.066 1.240 1.538 2.217 3.639 4.028 4.851
D 0.0278 0.0473 0.0354 0.0297 0.0197 0.0378 0.0450 0.0416
(14) (9) (13) (11) (9) (9) (9) (9)
d 0.0214 0.0182 0.028 0.0164 0.0063 0.0132 0.0166 0.0130
(9) (9) (8) (9) (9) (9) (9) (9)

8
 D 6300 – 03
TABLE 7 Example Statistics Indicating Need to Reject an Entire Sample
Sample number 90 89 93 92 91 94 95 96
m 96.1 99.8 119.3 125.4 126.0 139.9 139.4 159.5
D 5.10 4.20 15.26 4.40 4.09 4.87 4.74 3.85
(8) (9) (8) (11) (10) (8) (9) (8)
d 1.13 0.99 2.97 0.91 0.73 1.32 1.12 1.36
(8) (8) (8) (8) (8) (8) (8) (8)

of freedom, is approximately 4. The test ratio greatly exceeds 7.5.3.1 The two results from Laboratory D on Sample 1
this and results from Sample 93 shall therefore be rejected. were rejected (see 7.3.4) and thus a41 has to be estimated.
7.4.5.8 Turning to repeats standard deviations, it is noted Total of remaining results in Laboratory 4 = 36.354
that degrees of freedom are identical for each sample and that Total of remaining results in Sample 1 = 19.845
Total of all the results except a41= 348.358
Cochran’s test can therefore be applied. Cochran’s criterion Also S8 = 8 and L = 9.
will be the ratio of the largest sum of squares (Sample 93) to
the sum of all the sums of squares, that is Hence, the estimate of a41 is given by
1
2.972/~1.13210.9921...11.36 2! 5 0.510 (9) aij 5 @~9 3 36.354! 1 ~8 3 19.845! – 348.358# (11)
~9–1! ~8–1!
This is greater than the critical value of 0.352 corresponding to
Therefore,
n = 8 and v = 8 (see Table A2.2), and confirms that results from
Sample 93 shall be rejected. 137.588
aij 5 56 5 2.457 (12)
7.5 Estimating Missing or Rejected Values:
7.5.1 One of the Two Repeat Values Missing or Rejected—If 7.6 Rejection Test for Outlying Laboratories:
one of a pair of repeats (Yij1 or Yij2) is missing or rejected, this 7.6.1 At this stage, one further rejection test remains to be
shall be considered to have the same value as the other repeat carried out. This determines whether it is necessary to reject the
in accordance with the least squares method. complete set of results from any particular laboratory. It could
7.5.2 Both Repeat Values Missing or Rejected: not be carried out at an earlier stage, except in the case where
7.5.2.1 If both the repeat values are missing, estimates of aij no individual results or pairs are missing or rejected. The
(= Yij1+Yij2) shall be made by forming the laboratories 3 procedure again consists of Hawkins’ test (see 7.3.4), applied
samples interaction sum of squares (see Eq 17), including the to the laboratory averages over all samples, with any estimated
missing values of the totals of the laboratories/samples pairs of results included. If any laboratories are rejected on all samples,
results as unknown variables. Any laboratory or sample from new estimates shall be calculated for any remaining missing
which all the results were rejected shall be ignored and new values (see 7.5).
values of L and S used. The estimates of the missing or rejected 7.6.2 Worked Example:
values shall be those that minimize the interaction sum of 7.6.2.1 The procedure on the laboratory averages shown in
squares. Table 8 follows exactly that specified in 7.3.4. The deviations
7.5.2.2 If the value of single pair sum aij has to be estimated, of laboratory averages from the overall mean are given in Table
the estimate is given by the equation: 9 in units of the third decimal place, together with the sum of
squares. Hawkins’ test ratio is therefore:
1
aij 5 ~LL1 1 S8S1 – T1! (10) B* 5 0.026/=0.00222 5 0.5518
~L–1! ~S8–1! (13)
Comparison with the value tabulated in Table A1.5, for n = 9
where:
L1 = total of remaining pairs in the ith laboratory, and v = 0, shows that this ratio is not significant and therefore
S1 = total of remaining pairs in the jth sample, no complete laboratory rejections are necessary.
S8 = S – number of samples rejected in 7.4, and 7.7 Confirmation of Selected Transformation:
T1 = total of all pairs except aij. 7.7.1 At this stage it is necessary to check that the rejections
7.5.2.3 If more estimates are to be made, the technique of carried out have not invalidated the transformation used. If
successive approximation can be used. In this, each pair sum is necessary, the procedure from 7.2 shall be repeated with the
estimated in turn from Eq 10, using L1, S1, and T1, values, outliers replaced, and if a new transformation is selected,
which contain the latest estimates of the other missing pairs. outlier tests shall be reapplied with the replacement values
Initial values for estimates can be based on the appropriate reestimated, based on the new transformation.
sample mean, and the process usually converges to the required 7.7.2 Worked Example:
level of accuracy within three complete iterations (5). 7.7.2.1 It was not considered necessary in this case to repeat
7.5.3 Worked Example: the calculations from 7.2 with the outlying pair deleted.

TABLE 8 Averages of All Transformed Results from Each Laboratory

Grand
Laboratory A B C D E F G H J
Average
Average 2.437 2.439 2.424 2.426A 2.444 2.458 2.410 2.428 2.462 2.436
A
Including estimated value.

9
 D 6300 – 03
TABLE 9 Absolute Deviations of Laboratory Averages from Grand Average 3 1000
Sum of
Laboratory A B C D E F G H J
Squares
Deviation 1 3 12 10 8 22 26 8 26 2.22

8. Analysis of Variance and Calculation of Precision Samples sum of squares

Estimates
22.3022 1 72.512 2 1 ... 1 19.1922
8.1 After the data have been inspected for uniformity, a 5 – 854.6605
18
transformation has been performed, if necessary, and any (20)
outliers have been rejected (see Section 7), an analysis of
5 293.5409
variance shall be carried out. First an analysis of variance table
shall be constructed, and finally the precision estimates de-
rived. Laboratories sum of squares
8.2 Analysis of Variance:
38.9922 1 39.0202 1 ... 1 39.3872
8.2.1 Forming the Sums of Squares for the Laboratories 3 5 16
Samples Interaction Sum of Squares—The estimated values, if – 854.6605 (21)
any, shall be put in the array and an approximate analysis of
5 0.0356
variance performed.
M 5 mean correction 5 T2/2L8S8 (14)
Pairs sum of squares 5 ~1/2! ~2.5202 1 8.0412 1 ...
where: 1 2.2382! – 854.6605 (22)
L8 = L – number of laboratories rejected in 7.6 – number of 5 293.6908
laboratories with no remaining results after rejections
in 7.3.4, Repeats sum of squares 5 ~1/2! ~0.0422 1 0.0212 1 ... 1 02!
S8 = total of remaining pairs in the jth sample, and (23)
T = the total of all duplicate test results.
5 0.0219
S8
Samples sum of squares 5 @ ( ~g2j /2L8!# –M (15) Table 10 can then be derived.
j51
8.2.2 Forming the Sum of Squares for the Exact Analysis of
where gj is the sum of sample j test results. Variance:
L8 8.2.2.1 In this subsection, all the estimated pairs are disre-
Laboratories sum of squares 5 @ ( ~h2i /2S8!# – M (16) garded and new values of gj are calculated. The following sums
i51
of squares for the exact analysis of variance (6) are formed.
where hi is the sum of laboratory i test results.
S8
g2
( j
L8 S8
Uncorrected sample sum of squares 5 (24)
Pairs sum of squares 5 ~1/2! @ ( ( a2ij# –M (17) j51 Sj
i51 j51

where:
I = Laboratories 3 samples interaction sum of squares Sj = 2(L8 – number of missing pairs in that sample).
= (pairs sum of squares) – (laboratories sum of squares) L8 S8

– (sample sum of squares) Uncorrected pairs sum of squares 5 ~1/2! ( ( a2ij

i51 j51
(25)
Ignoring any pairs in which there are estimated values, The laboratories sum of squares is equal to (pairs sum of
repeats sum of squares, squares) – (samples sum of squares) – (the minimized labora-
L8 S8 tories 3 samples interaction sum of squares)
E 5 ~1/2! ( ( e2ij
F G
(18)
i51 j51 L8 S8 S8
g2
The purpose of performing this approximate analysis of 5 ~1/2! @ ( ( a2ij# – j51
( Sjj –I (26)
i51 j51
variance is to obtain the minimized laboratories 3 samples 8.2.2.2 Worked Example:
interaction sum of squares, I. This is then used as indicated in
8.2.2, to obtain the laboratories sum of squares. If there were Uncorrected samples sum of squares
no estimated values, the above analysis of variance is exact and
paragraph 8.2.2 shall be disregarded.
8.2.1.1 Worked Example: TABLE 10 Sums of Squares: Bromine Example
Sources of Variation Sum of Squares
350.8152
Mean correction 5 144 (19) Samples 293.5409
Laboratories 0.0356
5 854.6605 Laboratories 3 samples interaction 0.1143
Pairs 293.6908
where 854.6605 is the result obtained by electronic calculation Repeats 0.0219
without rounding the factors in the expression.

10
 D 6300 – 03
19.8452 72.5122 19.1922 TABLE 12 Analysis of Variance Table: Transformed Benzene
5 16 1 18 1 ... 1 18 (27) Example
Sum of Degrees of
5 1145.1834 Source of Variation
Squares Freedom
Mean Square F

Laboratories 0.0352 8 0.004400 2.117

2.5202 8.0412 2.2382
Uncorrected pairs sum of squares 5 2 1 2 1 ... 1 2 Laboratories 3 0.1143 55 0.002078
samples
(28)
5 1145.3329 Repeats 0.0219 71 0.000308 ...

Therefore, laboratories sum of squares

5 1145.3329– 1145.1834 1 0.1143 (29) 8.3 Expectation of Mean Squares and Calculation of Preci-
5 0.0352 sion Estimates:
8.3.1 Expectation of Mean Squares with No Estimated
8.2.3 Degrees of Freedom:
Values—For a complete array with no estimated values, the
8.2.3.1 The degrees of freedom for the laboratories are
expectations of mean squares are
(L8–1). The degrees of freedom for laboratories 3 samples
Laboratories: so2 + 2s12+ 2S8 s22
interaction are (L8–1)(S8–1) for a complete array and are Laboratories 3 samples: so2+ 2s12
reduced by one for each pair which is estimated. The degrees Repeats: so2
of freedom for repeats are (L8S8) and are reduced by one for
each pair in which one or both values are estimated. where:
8.2.3.2 Worked Example—There are eight samples and nine s12 = the component of variance due to interaction be-
laboratories in this example. As no complete laboratories or tween laboratories and samples, and
s22 = the component of variance due to differences be-
samples were rejected, then S8 = 8 and L8 = 9.
tween laboratories.
Laboratories degrees of freedom = L–1 = 8.
8.3.2 Expectation of Mean Squares with Estimated Values:
8.3.2.1 The coefficients of s12 and s22 in the expectation of
Laboratories 3 samples interaction degrees of freedom if there
mean squares are altered in the cases where there are estimated
had been no estimates, would have been (9–1)(8–1) = 56. But
values. The expectations of mean squares then become
one pair was estimated, hence laboratories 3 samples interac-
Laboratories: aso2+ 2s12+ b s22
tion degrees of freedom = 55. Repeats degrees of freedom Laboratories 3 samples:gs o2+ 2s12
would have been 72 if there had been no estimates. In this case Repeats: so2
one pair was estimated, hence repeats degrees of freedom = 71.
where:
8.2.4 Mean Squares and Analysis of Variance:
8.2.4.1 The mean square in each case is the sum of squares K – S8
b 5 2 L8 – 18 (30)
divided by the corresponding degrees of freedom. This leads to
the analysis of variance shown in Table 11. The ratio ML/MLS where:
is distributed as F with the corresponding laboratories and K = the number of laboratory 3 sample cells containing at
interaction degrees of freedom (see A1.7). If this ratio exceeds least one result, and a and g are computed as in 8.3.2.5
the 5 % critical value given in Table A1.6, then serious bias 8.3.2.2 If there are no cells with only a single estimated
between the laboratories is implied and the program organizer result, then a = g = 1.
shall be informed (see 6.5); further standardization of the test 8.3.2.3 If there are no empty cells (that is, every lab has
method may be necessary, for example, by using a certified tested every sample at least once, and K = L83 S8), then a and
reference material. g are both one plus the proportion of cells with only a single
8.2.4.2 Worked Example— The analysis of variance is result.
shown in Table 12. The ratio ML/MLS = 0.0044/0.002078 has a 8.3.2.4 If there are both empty cells and cells with only one
value 2.117. This is greater than the 5 % critical value obtained result, then, for each lab, compute the proportion of samples
from Table A1.6, indicating bias between laboratories. tested for which there is only one result, pi, and the sum of
these proportions over all labs, P. For each sample, compute
TABLE 11 Analysis of Variance Table the proportion of labs that have tested the sample for which
Mean there is only one result on it, qj, and the sum of these
Sources of Variation Degrees of Freedom Sum of Squares
Square proportions over samples, Q. Compute the total number of cells
Laboratories L8 − 1 Laboratories sum of ML with only one result, W, and the proportion of these among all
squares nonempty cells, W/K. Then
Laboratories 3 (L8 − 1) (S8 − 1) − number I MLS P – W/K
samples of estimated pairs a 5 1 1 L8–1 (31)

Repeats L8S8 − number of pairs in E Mr and

which one or both values
are estimated W – P – Q 1 W/K
g511 K – L8–S811 (32)

11
 D 6300 – 03
NOTE 5—These subsections are based upon the assumptions that both ~Reproducibility variance!2
samples and laboratories are random effects. v5 (39)
r12 r22 r32
8.3.2.5 Worked Example—For the example, which has eight L8 – 1 vLS 1 vr
1

samples and nine laboratories, one cell is empty (Laboratory D

where:
on Sample 1), so K = 71 and r1, r2, and r3 = the three successive terms in Eq 38,
71 – 8 vLS = the degrees of freedom for laboratories 3
b52 5 15.75 (33)
~9 – 1! samples, and
vr = the degrees of freedom for repeats.
None of the nonempty cells has only one result, so a = g =
1. To make the example more interesting, assume that only one (1) Round calculated estimates of reproducibility in accor-
result remains from Laboratory A on Sample 1. Then W = 1, p1 dance with Practice E 29, specifically paragraph 7.6 of that
= 1⁄8 , p2= p3= ... = p9= 0, and P = 0.125. We compute q1= 1⁄8 practice.
(we don’t count Laboratory D in the denominator), q2= q3=...= (2) Substantial bias between laboratories will result in a loss
q8= 0, and Q = 0.125. Consequently, of degrees of freedom estimated by Eq 39. If reproducibility
degrees of freedom are less than 30, then the program organizer
0.125 – 1/71 shall be informed (see 6.5); further standardization of the test
a511 9–1 5 1.014 (34)
method may be necessary.
and 8.3.3.4 Worked Example—Recalling that a = g = 1 (not
1 – 0.125 – 0.125 1 1/71 1.014, as shown in Eq 34 and 35):
g511 55 5 1.014 (35)
Reproducibility variance (40)
8.3.3 Calculation of Precision Estimates:
8.3.3.1 Repeatability—The repeatability variance is twice
S 2
D S
13.75
5 15.75 3 0.00440 1 15.75 3 0.002078 1 0.000308 D
the mean square for repeats. The repeatability estimate is the 5 0.000559 1 0.001814 1 0.000308
product of the repeatability standard deviation and the “t- 5 0.002681
value” with appropriate degrees of freedom (see Table A2.3)
corresponding to a two-sided probability of 95 %. Round 0.0026812
calculated estimates of repeatability in accordance with Prac- v5 2 (41)
0.000559 0.0018142 0.0003082
tice E 29, specifically paragraph 7.6 of that practice. Note that 8 1 55 1 71
if a transformation y = f(x) has been used, then 5 72

U U
dx
r~x! ' dy r~y! (36)
Reproducibility of y 5 t72= 0.002681 (42)
where r(x), r(y) are the corresponding repeatability functions 5 0.1034
(see). A similar relationship applies to the reproducibility
Reproducibility of x 5 0.310x2/3
functions R(x), R(y).
8.3.3.5 Determinability—When determinability is relevant,
8.3.3.2 Worked Example: it shall be calculated by the same procedure as is used to
Repeatability variance 5 2so2 (37) calculate repeatability except that pairs of determined values
5 0.000616 replace test results. This will as much as double the number of
“laboratories” for the purposes of this calculation.
Repeatability of y 5 t71=0.000616
8.3.4 Bias:
5 1.994 x 0.0428 8.3.4.1 Bias equals average sample test result minus its
5 0.0495 accepted reference value. In the ideal case, average 30 or more
Repeatability of x 5 3x2/3 3 0.0495 test results, measured independently by processes in a state of
statistical control, for each of several relatively uniform mate-
5 0.148x2/3 rials, the reference values for which have been established by
8.3.3.3 Reproducibility—Reproducibility variance = 2 one of the following alternatives, and subtract the reference
(so2+ s12+ s22) and can be calculated using Eq 38. values. In practice, the bias of the test method, for a specific
material, may be calculated by comparing the sample average
Reproducibility variance (38)
with the accepted reference value.
2
S D2
S 2
5 b ML 1 1 – b MLS 1 2 – g 1 b ~g – a! Mr 8.3.4.2 Accepted reference values may be one of the fol-
lowing: an assigned value for a Standard Reference Material, a
where the symbols are as set out in 8.2.4 and 8.3.2. The consensus value based on collaborative experimental work
reproducibility estimate is the product of the reproducibility under the guidance of a scientific or engineering organization,
standard deviation and the “t-value” with appropriate degrees an agreed upon value obtained using an accepted reference
of freedom (see Table A2.3), corresponding to a two-sided method, or a theoretical value.
probability of 95 %. An approximation (7) to the degrees of 8.3.4.3 Where possible, one or more materials with ac-
freedom of the reproducibility variance is given by Eq 39. cepted reference values shall be included in the interlaboratory

12
 D 6300 – 03
program. In this way sample averages free of outliers will method, exceed the following value in only one case in 20.
become available for use in determining bias. When this occurs, the operator must take corrective action:
8.3.4.4 Because there will always be at least some bias Determinability 5 0.59=m (45)
because of the inherent variability of test results, it is recom-
mended to test the bias value by applying Student’s t test using where m is the mean of the two determined values in mL.
the number of laboratories degrees of freedom for the sample 8.4.2 A graph or table may be used instead of, or in addition
made available during the calculation of precision. When the to, the equation format shown above. In any event, it is helpful
calculated t is less than the critical value at the 5 % confidence to include a table of typical values like Table 13.
level, the bias should be reported as not significant. 8.4.3 The wording to be used for test methods where the
8.4 Precision and Bias Section for a Test Method—When statistical treatment applied is unknown is: “The precision of
the precision of a test method has been determined, in this test is not known to have been obtained in accordance with
accordance with the procedures set out in this practice, it shall currently accepted guidelines (for example, in Committee D02,
be included in the test method as illustrated in these examples: Practice D 6300).” The existing statement of precision would
8.4.1 Precision—The precision of this test method, which then follow.
was determined by statistical examination of interlaboratory 8.5 Data Storage:
results using Practice D 6300, is as follows. 8.5.1 The interlaboratory program data should be preserved
8.4.1.1 Repeatability—The difference between successive for general reference. Prepare a research report containing
results obtained by the same operator with the same apparatus details of the test program, including description of the
under constant operating conditions on identical test material samples, the raw data, and the calculations described herein.
would in the long run, in the normal and correct operation of Send the file to ASTM Headquarters and request a File
the test method exceed the following values only in one case in Reference Number.
20. 8.5.2 Use the following footnote style in the precision
section of the test method. “The results of the cooperative test
Repeatability 5 0.148 x2/3 (43)
program, from which these values have been derived, are filed
where x is the average of two results. at ASTM Headquarters as RR:D02–XXXX.”
8.4.1.2 Reproducibility—The difference between two single
and independent results obtained by different operators work- 9. Keywords
ing in different laboratories on identical test material would in 9.1 interlaboratory; precision; repeatability; reproducibility;
the long run exceed the following values only in one case in 20. round robin
Reproducibility 5 0.310 x2/3 (44)
TABLE 13 Typical Precision Values: Bromine Example
where x is the average of two results. Average Value Repeatability Reproducibility
8.4.1.3 If determinability is relevant, it shall precede repeat- Bromine Numbers Bromine Numbers Bromine Numbers
ability in the statement above. The unit of measurement shall 1.0 0.15 0.31
be specified when it differs from that of the test result: 2.0 0.23 0.49
8.4.1.4 Determinability—The difference between the pair of 10.0 0.69 1.44
20.0 1.09 2.28
determined values averaged to obtain a test result would, in the 100.0 3.19 6.68
long run, in the normal and correct operation of the test

ANNEXES

(Mandatory Information)

A1. NOTATION AND TESTS

A1.1 The Following Notation Is Used Throughout This m = the mean of sample test results,
Practice: x = the mean of a pair of test results in repeatability and
reproducibility statements,
x... = an individual test result,
a = the sum of duplicate test results, y... = a transformed value of x..., and
e = the difference between duplicate test results, v = the degrees of freedom.
g = the sum of sample test results,
h = the sum of laboratory test results, A1.2 Array of Duplicate Results from Each of L
i = the suffix denoting laboratory number,
Laboratories on S Samples and Corresponding
j = the suffix denoting sample number,
S = the number of samples, Means mj
T = the total of all duplicate test results, A1.2.1 See Table A1.1.
L = the number of laboratories, NOTE A1.1—If a transformation y = F(x) of the reported data is

13
 D 6300 – 03
TABLE A1.1 Typical Layout of Data from Round Robin 1
D2j 5 K @C2j 1 ~Kj – 1! d2j # (A1.3)
Sample j
Laboratory 1 2 j S
where:
1 x111 x121 x1j1 x1S1
L
x112 x122 x1j2 x1S2
Kj 5 ~S2j – ( n2ij! / @Sj ~L–1!#
i51
(A1.4)
2 x211 x221 x2j1 x2S1
x212 x222 x2j2 x2S2

i xi11 xi21 xij1 xiS1 nij = number of results obtained by Laboratory i from
xi12 xi22 xij2 xiS2 Sample j,
Sj = total number of results obtained from Sample j, and
L xL11 xL21 xLj1 xLS1
xL12 xL22 xLj2 xLS2
L = number of cells in Sample j containing at least one
result.
Total g1 g2 gj gs A1.4.4 Laboratories degrees of freedom for Sample j is
Mean m1 m2 mj ms given approximately (6) by:
~KjD2j !2
vj 5 (A1.5)
~C2j !2 @~Kj–1!d2j #2
necessary (see 7.2), then corresponding symbols yij1 and yij2 are used in L–1 1 L
place of xij1 and xij2.
(rounded to the nearest integer)
A1.3 Array of Sums of Duplicate Results, of Laboratory A1.4.5 If either or both of a laboratory/sample pair of results
Totals hi and Sample Totals gj is missing, the factor L is reduced by one.
A1.3.1 See Table A1.2. A1.4.6 If both of a laboratory/sample pair of results is
A1.3.2 If any results are missing from the complete array, missing, the factor (L – 1) is reduced by one.
then the divisor in the expression for mj will be correspond-
ingly reduced. A1.5 Cochran’s Test
A1.5.1 The largest sum of squares, SSk, out of a set of n
A1.4 Sums of Squares and Variances (7.2) mutually independent sums of squares each based on v degrees
A1.4.1 Repeats Variance for Sample j: of freedom, can be tested for conformity in accordance with:
L SSk
( e2ij
i51
Cochran’s criterion 5 n (A1.6)
d2j 5 2L (A1.1) ( SSi
i51

where: A1.5.2 The test ratio is identical if sum of squares values are
L = the repeats degrees of freedom for Sample j, one degree replaced by mean squares (variance estimates). If the calcu-
of freedom for each laboratory pair. If either or both of lated ratio exceeds the critical value given in Table A1.3, then
a laboratory/sample pair of results is missing, the the sum of squares in question, SSk, is significantly greater than
corresponding term in the numerator is omitted and the the others with a probability of 99 %. Examples of SSi include
factor L is reduced by one. eij2 and dj2(Eq A1.1).
A1.4.2 Between Cells Variance for Sample j:

F G
A1.6 Hawkins’ Test
L
a2ij g2j
C2j 5 (
i51 nij
– S /~L–1!
j
(A1.2) A1.6.1 An extreme value in a data set can be tested as an
outlier by comparing its deviation from the mean value of the
A1.4.3 Laboratories Variance for Sample j: data set to the square root of the sum of squares of all such
deviations. This is done in the form of a ratio. Extra informa-
TABLE A1.2 Typical Layout of Sums of Duplicate ResultsA tion on variability can be provided by including independent
Sample sums of squares into the calculations. These will be based on v
Laboratory 1 2 j S Total degrees of freedom and will have the same population variance
1 a11 a12 a1j aiS h1 as the data set in question. Table A1.4 shows the values that are
2 a21 a22 a2j a2S h2
i ai1 ai 2 aij ai 1 hi
required to apply Hawkins’ test to individual samples. The test
L aL1 aL2 aLj aLS hL procedure is as follows:
A1.6.1.1 Identify the sample k and cell mean aik/nik, which
Total g1 g2 gj gS T
A
has the most extreme absolute deviation ?aik/nik – mk? . The cell
aij = xij1 + xij2 (or aij = yij1 + y ij2, if a transformation has been used)
eij = x ij1 – x ij2 (or aij = yij1 – yij2, if a transformation has been used)
identified will be the candidate for the outlier test, be it high or
low.
L S
gj 5 ( aij hi 5 ( aij A1.6.1.2 Calculate the total sum of squares of deviations
i51 j51
S
mj 5 gj / 2L L S SS 5 ( SSj (A1.7)
T5 ( hi 5 j(
i51 51
gj i51

A1.6.1.3 Calculate the test ratio

14
 D 6300 – 03
TABLE A1.3 Cube Root of Bromine Number for Low Boiling Samples
Sample
Laboratory 1 2 3 4 5 6 7 8
A 1.239 4.010 0.928 1.547 2.224 3.586 4.860 1.063
1.281 4.031 0.921 1.560 2.231 3.596 4.852 1.063

B 1.193 4.029 0.884 1.547 2.231 3.691 4.856 1.063

1.216 4.041 0.896 1.547 2.224 3.682 4.853 1.063

C 1.216 3.990 0.913 1.518 2.183 3.647 4.826 1.091

1.216 3.996 0.913 1.518 2.190 3.639 4.830 1.091

D 1.601 3.992 0.928 1.587 2.210 3.674 4.774 1.000

1.578 3.998 0.928 1.574 2.210 3.682 4.765 1.032

E 1.281 3.998 0.940 1.547 2.217 3.619 4.871 1.091

1.216 3.994 0.940 1.547 2.231 3.624 4.864 1.119

F 1.216 4.135 0.896 1.504 2.257 3.662 4.946 1.119

1.193 4.115 0.862 1.533 2.237 3.632 4.903 1.119

G 1.239 3.996 0.917 1.518 2.197 3.586 4.850 1.032

1.301 3.992 0.839 1.518 2.197 3.570 4.832 0.976

H 1.260 4.051 0.921 1.474 2.204 3.674 4.860 1.032

1.216 4.031 0.892 1.518 2.204 3.647 4.856 1.000

J 1.281 4.086 0.932 1.587 2.231 3.662 4.873 1.119

1.281 4.027 0.932 1.547 2.231 3.632 4.847 1.119

TABLE A1.4 Calculations for Hawkins’ Test for OutliersA containing:

Sample n = number of laboratories = L,
1 2 j S m = overall mean = T/N, where N is the total number of results
No. of cells n1 n2 nj ns in the array, and
Sample mean m1 m2 mj ms
Sum of squares SS1 SS2 SSj SSs
SS = sum of squares of deviations of laboratory means from the
A
overall mean, and is given by
nj = the number of cells in Sample j which contains at least one result,
mj = the mean of Sample j, and
SSj = the sum of squares of deviations of cell means a ij/nij from sample mean
m j, and is given by
SS 5 (
i51
L
S D
hi
ni – m
2
(A1.10)

SS j 5 ~L – 1! C2j
(L–1) is the between cells (laboratories) degrees of freedom, and shall be where:
reduced by 1 for every cell in Sample j which does not contain a result. ni = the number of results in Laboratory i.
In the test procedure, therefore, identify the laboratory mean
?aik/nik – mk? hi/ni which differs most from the overall mean, m. The
B* 5 (A1.8)
=SS corresponding test ratio then becomes:
A1.6.1.4 Compare the test ratio with the critical value from ? hi/ni – m?
Table A1.5, for n = nk and extra degrees of freedom v where B* 5 (A1.11)
= SS
S
A1.6.1.7 This shall be compared with the critical value from
v5 ( ~nj – 1!, j fi k.
j51
(A1.9)
Table A1.5 as before, but now with extra degrees of freedom v
A1.6.1.5 If B* exceeds the critical value, reject results from = 0. If a laboratory is rejected, adjust the values of n, m, and SS
the cell in question (Sample k, Laboratory i), modify nk, mk and accordingly and repeat the calculations.
SSk values accordingly, and repeat from A1.6.1.1.
A1.7 Variance Ratio Test (F-Test)
NOTE A1.2—Hawkins’ test applies theoretically to the detection of only
a single outlier laboratory in a sample. The technique of repeated tests for A1.7.1 A variance estimate V1, based on v1 degrees of
a single outlier, in the order of maximum deviation from sample mean, freedom, can be compared with a second estimate V2, based on
implies that the critical values in Table A1.5 will not refer exactly to the v2 degrees of freedom, by calculating the ratio
1 % significance level. It has been shown by Hawkins, however, that if n
$ 5 and the total degrees of freedom (n + v) are greater than 20, then this V1
F5V (A1.12)
effect is negligible, as are the effects of masking (one outlier hiding 2
another) and swamping (the rejection of one outlier leading to the A1.7.2 If the ratio exceeds the appropriate critical value
rejection of others). given in Tables A1.6-A1.9, where v1 corresponds to the
A1.6.1.6 When the test is applied to laboratories averaged numerator and v2 corresponds to the denominator, then V1 is
over all samples, Table A1.4 will reduce to a single column greater than V2 at the chosen level of significance.

15
 D 6300 – 03
TABLE A1.5 Critical Values of Hawkins’ 1 % Outlier Test for n = 3 to 50 and y = 0 to 200
Degrees of Freedom y
n 0 5 10 15 20 30 40 50 70 100 150 200
3 0.8165 0.7240 0.6100 0.5328 0.4781 0.4049 0.3574 0.3233 0.2769 0.2340 0.1926 0.1674
4 0.8639 0.7505 0.6405 0.5644 0.5094 0.4345 0.3850 0.3492 0.3000 0.2541 0.2096 0.1824
5 0.8818 0.7573 0.6530 0.5796 0.5258 0.4510 0.4012 0.3647 0.3142 0.2668 0.2204 0.1920
6 0.8823 0.7554 0.6571 0.5869 0.5347 0.4612 0.4115 0.3749 0.3238 0.2755 0.2280 0.1988
7 0.8733 0.7493 0.6567 0.5898 0.5394 0.4676 0.4184 0.3819 0.3307 0.2819 0.2337 0.2039
8 0.8596 0.7409 0.6538 0.5901 0.5415 0.4715 0.4231 0.3869 0.3358 0.2868 0.2381 0.2079
9 0.8439 0.7314 0.6493 0.5886 0.5418 0.4738 0.4262 0.3905 0.3396 0.2906 0.2416 0.2112
10 0.8274 0.7213 0.6439 0.5861 0.5411 0.4750 0.4283 0.3930 0.3426 0.2936 0.2445 0.2139
11 0.8108 0.7111 0.6380 0.5828 0.5394 0.4753 0.4295 0.3948 0.3448 0.2961 0.2469 0.2162
12 0.7947 0.7010 0.6318 0.5790 0.5373 0.4750 0.4302 0.3960 0.3466 0.2981 0.2489 0.2181
13 0.7791 0.6910 0.6254 0.5749 0.5347 0.4742 0.4304 0.3968 0.3479 0.2997 0.2507 0.2198
14 0.7642 0.6812 0.6189 0.5706 0.5319 0.4731 0.4302 0.3972 0.3489 0.3011 0.2521 0.2212
15 0.7500 0.6717 0.6125 0.5662 0.5288 0.4717 0.4298 0.3973 0.3496 0.3021 0.2534 0.2225
16 0.7364 0.6625 0.6061 0.5617 0.5256 0.4701 0.4291 0.3972 0.3501 0.3030 0.2544 0.2236
17 0.7235 0.6535 0.5998 0.5571 0.5223 0.4683 0.4282 0.3968 0.3504 0.3037 0.2554 0.2246
18 0.7112 0.6449 0.5936 0.5526 0.5189 0.4665 0.4272 0.3964 0.3505 0.3043 0.2562 0.2254
19 0.6996 0.6365 0.5876 0.5480 0.5155 0.4645 0.4260 0.3958 0.3506 0.3047 0.2569 0.2262
20 0.6884 0.6286 0.5816 0.5436 0.5120 0.4624 0.4248 0.3951 0.3505 0.3051 0.2575 0.2269
21 0.6778 0.6209 0.5758 0.5392 0.5086 0.4603 0.4235 0.3942 0.3503 0.3053 0.2580 0.2275
22 0.6677 0.6134 0.5702 0.5348 0.5052 0.4581 0.4221 0.3934 0.3500 0.3055 0.2584 0.2280
23 0.6581 0.6062 0.5647 0.5305 0.5018 0.4559 0.4206 0.3924 0.3496 0.3056 0.2588 0.2285
24 0.6488 0.5993 0.5593 0.5263 0.4984 0.4537 0.4191 0.3914 0.3492 0.3056 0.2591 0.2289
25 0.6400 0.5925 0.5540 0.5221 0.4951 0.4515 0.4176 0.3904 0.3488 0.3056 0.2594 0.2293
26 0.6315 0.5861 0.5490 0.5180 0.4918 0.4492 0.4160 0.3893 0.3482 0.3054 0.2596 0.2296
27 0.6234 0.5798 0.5440 0.5140 0.4885 0.4470 0.4145 0.3881 0.3477 0.3053 0.2597 0.2299
28 0.6156 0.5737 0.5392 0.5101 0.4853 0.4447 0.4129 0.3870 0.3471 0.3051 0.2599 0.2302
29 0.6081 0.5678 0.5345 0.5063 0.4821 0.4425 0.4113 0.3858 0.3464 0.3049 0.2600 0.2304
30 0.6009 0.5621 0.5299 0.5025 0.4790 0.4403 0.4097 0.3846 0.3458 0.3047 0.2600 0.2306
35 0.5686 0.5361 0.5086 0.4848 0.4641 0.4294 0.4016 0.3785 0.3421 0.3031 0.2600 0.2312
40 0.5413 0.5136 0.4897 0.4688 0.4504 0.4191 0.3936 0.3722 0.3382 0.3010 0.2594 0.2314
45 0.5179 0.4939 0.4728 0.4542 0.4377 0.4094 0.3859 0.3660 0.3340 0.2987 0.2586 0.2312
50 0.4975 0.4764 0.4577 0.4410 0.4260 0.4002 0.3785 0.3600 0.3299 0.2962 0.2575 0.2308

TABLE A1.6 Critical 5 % Values of F

y1
3 4 5 6 7 8 9 10 15 20 30 50 100 200 500 `
3 9.28 9.12 9.01 8.94 8.89 8.85 8.81 8.79 8.70 8.66 8.62 8.58 8.55 8.54 8.53 8.53
4 6.59 6.39 6.26 6.16 6.09 6.04 6.00 5.96 5.86 5.80 5.75 5.70 5.66 5.65 5.64 5.63
5 5.41 5.19 5.05 4.95 4.88 4.82 4.77 4.74 4.62 4.56 4.50 4.44 4.41 4.39 4.37 4.37
6 4.76 4.53 4.39 4.28 4.21 4.15 4.10 4.06 3.94 3.87 3.81 3.75 3.71 3.69 3.68 3.67
7 4.35 4.12 3.97 3.87 3.79 3.73 3.68 3.64 3.51 3.44 3.38 3.32 3.27 3.25 3.24 3.23
8 4.07 3.84 3.69 3.58 3.50 3.44 3.39 3.35 3.22 3.15 3.08 3.02 2.97 2.95 2.94 2.93
9 3.86 3.63 3.48 3.37 3.29 3.23 3.18 3.14 3.01 2.94 2.86 2.80 2.76 2.73 2.72 2.71
10 3.71 3.48 3.33 3.22 3.14 3.07 3.02 2.98 2.85 2.77 2.70 2.64 2.59 2.56 2.55 2.54
y2
15 3.29 3.06 2.90 2.79 2.71 2.64 2.59 2.54 2.40 2.33 2.25 2.18 2.12 2.10 2.08 2.07
20 3.10 2.87 2.71 2.60 2.51 2.45 2.39 2.35 2.20 2.12 2.04 1.97 1.91 1.88 1.86 1.84
30 2.92 2.69 2.53 2.42 2.33 2.27 2.21 2.16 2.01 1.93 1.84 1.76 1.70 1.66 1.64 1.62
50 2.79 2.56 2.40 2.29 2.20 2.13 2.07 2.03 1.87 1.78 1.69 1.60 1.52 1.48 1.46 1.44
100 2.70 2.46 2.31 2.19 2.10 2.03 1.97 1.93 1.77 1.68 1.57 1.48 1.39 1.34 1.31 1.28
200 2.65 2.42 2.26 2.14 2.06 1.98 1.93 1.88 1.72 1.62 1.52 1.41 1.32 1.26 1.22 1.19
500 2.62 2.39 2.23 2.12 2.03 1.96 1.90 1.85 1.69 1.59 1.48 1.38 1.28 1.21 1.16 1.11
` 2.60 2.37 2.21 2.10 2.01 1.94 1.88 1.83 1.67 1.57 1.46 1.35 1.24 1.17 1.11 1.00

16
 D 6300 – 03
TABLE A1.7 Critical 1 % Values of F
y1
3 4 5 6 7 8 9 10 15 20 30 50 100 200 500 `
3 29.5 28.7 28.2 27.9 27.7 27.5 27.3 27.2 26.9 26.7 26.5 26.4 26.2 26.2 26.1 26.1
4 16.7 16.0 15.5 15.2 15.0 14.8 14.7 14.5 14.2 14.0 13.8 13.7 13.6 13.5 13.5 13.5
5 12.1 11.4 11.0 10.7 10.5 10.3 10.2 10.1 9.72 9.55 9.38 9.24 9.13 9.08 9.04 9.02
6 9.78 9.15 8.75 8.47 8.26 8.10 7.98 7.87 7.56 7.40 7.23 7.09 6.99 6.93 6.90 6.88
7 8.45 7.85 7.46 7.19 6.99 6.84 6.72 6.62 6.31 6.16 5.99 5.86 5.75 5.70 5.67 5.65
8 7.59 7.01 6.63 6.37 6.18 6.03 5.91 5.81 5.52 5.36 5.20 5.07 4.96 4.91 4.88 4.86
9 6.99 6.42 6.06 5.80 5.61 5.47 5.35 5.26 4.96 4.81 4.65 4.52 4.42 4.36 4.33 4.31
10 6.55 5.99 5.64 5.39 5.20 5.06 4.94 4.85 4.56 4.41 4.25 4.12 4.01 3.96 3.93 3.91
y2
15 5.42 4.89 4.56 4.32 4.14 4.00 3.89 3.80 3.52 3.37 3.21 3.08 2.98 2.92 2.89 2.87
20 4.94 4.43 4.10 3.87 3.70 3.56 3.46 3.37 3.09 2.94 2.78 2.64 2.54 2.48 2.44 2.42
30 4.51 4.02 3.70 3.47 3.30 3.17 3.07 2.98 2.70 2.55 2.39 2.25 2.13 2.07 2.03 2.01
50 4.20 3.72 3.41 3.19 3.02 2.89 2.79 2.70 2.42 2.27 2.10 1.95 1.82 1.76 1.71 1.68
100 3.98 3.51 3.21 2.99 2.82 2.69 2.59 2.50 2.22 2.07 1.89 1.73 1.60 1.52 1.47 1.43
200 3.88 3.41 3.11 2.89 2.73 2.60 2.50 2.41 2.13 1.97 1.79 1.63 1.48 1.39 1.33 1.28
500 3.82 3.36 3.05 2.84 2.68 2.55 2.44 2.36 2.07 1.92 1.74 1.56 1.41 1.31 1.23 1.16
` 3.78 3.32 3.02 2.80 2.64 2.51 2.41 2.32 2.04 1.88 1.70 1.52 1.36 1.25 1.15 1.00

TABLE A1.8 Critical 0.1 % Values of F

y1
3 4 5 6 7 8 9 10 15 20 30 50 100 200 500 `
3 141 137 135 133 132 131 130 129 127 126 125 125 124 124 124 124
4 56.2 53.4 51.7 50.5 49.7 49.0 48.5 48.0 46.8 46.1 45.4 44.9 44.5 44.3 44.1 44.0
5 33.2 31.1 29.8 28.8 28.2 27.6 27.2 26.9 25.9 25.4 24.9 24.4 24.1 23.9 23.8 23.8
6 23.7 21.9 20.8 20.0 19.5 19.0 18.7 18.4 17.6 17.1 16.7 16.3 16.0 15.9 15.8 15.8
7 18.8 17.2 16.2 15.5 15.0 14.6 14.3 14.1 13.3 12.9 12.5 12.2 11.9 11.8 11.7 11.7
8 15.8 14.4 13.5 12.9 12.4 12.0 11.8 11.5 10.8 10.5 10.1 9.80 9.57 9.46 9.39 9.34
9 13.9 12.6 11.7 11.1 10.7 10.4 10.1 9.89 9.24 8.90 8.55 8.26 8.04 7.93 7.86 7.81
10 12.6 11.3 10.5 9.92 9.52 9.20 8.96 8.75 8.13 7.80 7.47 7.19 6.98 6.87 6.81 6.76
y2
15 9.34 8.25 7.57 7.09 6.74 6.47 6.26 6.08 5.53 5.25 4.95 4.70 4.51 4.41 4.35 4.31
20 8.10 7.10 6.46 6.02 5.69 5.44 5.24 5.08 4.56 4.29 4.01 3.77 3.58 3.48 3.42 3.38
30 7.05 6.12 5.53 5.12 4.82 4.58 4.39 4.24 3.75 3.49 3.22 2.98 2.79 2.69 2.63 2.59
50 6.34 5.46 4.90 4.51 4.22 4.00 3.82 3.67 3.20 2.95 2.68 2.44 2.24 2.14 2.07 2.03
100 5.85 5.01 4.48 4.11 3.83 3.61 3.44 3.30 2.84 2.59 2.32 2.07 1.87 1.75 1.68 1.62
200 5.64 4.81 4.29 3.92 3.65 3.43 3.26 3.12 2.67 2.42 2.15 1.90 1.68 1.55 1.46 1.39
500 5.51 4.69 4.18 3.82 3.54 3.33 3.16 3.02 2.58 2.33 2.05 1.80 1.57 1.43 1.32 1.23
` 5.42 4.62 4.10 3.74 3.47 3.27 3.10 2.96 2.51 2.27 1.99 1.73 1.49 1.34 1.21 1.00

TABLE A1.9 Critical 0.05 % Values of F

y1
3 4 5 6 7 8 9 10 15 20 30 50 100 200 500 `
3 225 218 214 211 209 208 207 206 203 201 199 198 197 197 196 196
4 80.1 76.1 73.6 71.9 70.6 69.7 68.9 68.3 66.5 65.5 64.6 63.8 63.2 62.9 62.7 62.6
5 44.4 41.5 39.7 38.5 37.6 36.9 36.4 35.9 34.6 33.9 33.1 32.5 32.1 31.8 31.7 31.6
6 30.4 28.1 26.6 25.6 24.9 24.3 23.9 23.5 22.4 21.9 21.4 20.9 20.5 20.3 20.2 20.1
7 23.5 21.4 20.2 19.3 18.7 18.2 17.8 17.5 16.5 16.0 15.5 15.1 14.7 14.6 14.5 14.4
8 19.4 17.6 16.4 15.7 15.1 14.6 14.3 14.0 13.1 12.7 12.2 11.8 11.6 11.4 11.4 11.3
9 16.8 15.1 14.1 13.3 12.8 12.4 12.1 11.8 11.0 10.6 10.2 9.80 9.53 9.40 9.32 9.26
10 15.0 13.4 12.4 11.8 11.3 10.9 10.6 10.3 9.56 9.16 8.75 8.42 8.16 8.04 7.96 7.90
y2
15 10.8 9.48 8.66 8.10 7.68 7.36 7.11 6.91 6.27 5.93 5.58 5.29 5.06 4.94 4.87 4.83
20 9.20 8.02 7.28 6.76 6.38 6.08 5.85 5.66 5.07 4.75 4.42 4.15 3.93 3.82 3.75 3.70
30 7.90 6.82 6.14 5.66 5.31 5.04 4.82 4.65 4.10 3.80 3.48 3.22 3.00 2.89 2.82 2.78
50 7.01 6.01 5.37 4.93 4.60 4.34 4.14 3.98 3.45 3.16 2.86 2.59 2.37 2.25 2.17 2.13
100 6.43 5.47 4.87 4.44 4.13 3.89 3.70 3.54 3.03 2.75 2.44 2.18 1.95 1.82 1.74 1.67
200 6.16 5.23 4.64 4.23 3.92 3.68 3.49 3.34 2.83 2.56 2.25 1.98 1.74 1.60 1.50 1.42
500 6.01 5.09 4.51 4.10 3.80 3.56 3.36 3.21 2.72 2.45 2.14 1.87 1.61 1.46 1.34 1.24
` 5.91 5.00 4.42 4.02 3.72 3.48 3.30 3.14 2.65 2.37 2.07 1.79 1.53 1.36 1.22 1.00

17
 D 6300 – 03

A2. EXAMPLE RESULTS OF TEST FOR DETERMINATION OF BROMINE NUMBER AND STATISTICAL TABLES

A2.1 Bromine Number for Low Boiling Samples A2.5 Critical Values of t
A2.1.1 See Table A2.1. A2.5.1 See Table A2.3.
A2.2 Cube Root of Bromine Number for Low Boiling A2.6 Critical Values of F9
Samples
A2.6.1 Critical 5 % Values of F—See Table A1.6.
A2.2.1 See Table A1.3.
A2.6.2 Critical 1 % Values of F—See Table A1.7.
A2.3 Critical 1 % Values of Cochran’s Criterion for n A2.6.3 Critical 0.1 % Values of F—See Table A1.8.
Variance Estimates and v Degrees of Freedom A2.6.4 Critical 0.05 % Values of F—See Table A1.9.
A2.3.1 See Table A2.2. A2.6.5 Approximate Formula for Critical Values of
F—Critical values of F for untabulated values of v1, and v2
A2.4 Critical Values of Hawkins’ 1 % Outlier Test for n may be approximated by second order interpolation from the
= 3 to 50 and v = 0 to 200 tables. Critical values of F corresponding to v1 >30 and v2 >30
A2.4.1 See Table A1.5. degrees of freedom and significance level 100 (1–P) %, where
A2.4.2 The critical values in the table are correct to the P is the probability, can also be approximated from the formula
fourth decimal place in the range n = 3 to 30 and v = 0, 5, 15,
and 30 (3). Other values were derived from the Bonferroni log10 ~F! 5
A~P!
= b – B~P!
1
S1
– C~P! v 1 v
1 2
D (A2.2)
inequality as
where:

F G S D
1
~n–1! 2 1 1
B* 5 t (A2.1) b 5 2/ v 1 v (A2.3)
n ~ n 1 v – 2 1 t2 ! 1 2

where t is the upper 0.005/ n fractile of a t-variate with n + A2.6.5.1 Values of A (P), B (P), and C(P) are given in Table
v – 2 degrees of freedom. The values so computed are only A2.4 for typical values of significance level 100 (1–P) %.
slightly conservative, and have a maximum error of approxi-
mately 0.0002 above the true value. If critical values are A2.7 Critical Values of the Normal Distribution (see Table
required for intermediate values of n and v, they may be A2.5):
estimated by second order interpolation using the square of the
reciprocals of the tabulated values. Similarly, second order
extrapolation can be used to estimate values beyond n = 50 and
v = 200. 9
See (8) for the source of these tables.

TABLE A2.1 Bromine Number for Low Boiling Samples

Sample
Laboratory 1 2 3 4 5 6 7 8
A 1.9 64.5 0.80 3.7 11.0 46.1 114.8 1.2
2.1 65.5 0.78 3.8 11.1 46.5 114.2 1.2

B 1.7 65.4 0.69 3.7 11.1 50.3 114.5 1.2

1.8 66.0 0.72 3.7 11.0 49.9 114.3 1.2

C 1.8 63.5 0.76 3.5 10.4 48.5 112.4 1.3

1.8 63.8 0.76 3.5 10.5 48.2 112.7 1.3

D 4.1 63.6 0.80 4.0 10.8 49.6 108.8 1.0

4.0 63.9 0.80 3.9 10.8 49.9 108.2 1.1

E 2.1 63.9 0.83 3.7 10.9 47.4 115.6 1.3

1.8 63.7 0.83 3.7 11.1 47.6 115.1 1.4

F 1.8 70.7 0.72 3.4 11.5 49.1 121.0 1.4

1.7 69.7 0.64 3.6 11.2 47.9 117.9 1.4

G 1.9 63.8 0.77 3.5 10.6 46.1 114.1 1.1

2.2 63.6 0.59 3.5 10.6 45.5 112.8 0.93

H 2.0 66.5 0.78 3.2 10.7 49.6 114.8 1.1

1.8 65.5 0.71 3.5 10.7 48.5 114.5 1.0

J 2.1 68.2 0.81 4.0 11.1 49.1 115.7 1.4

2.1 65.3 0.81 3.7 11.1 47.9 113.9 1.4

18
 D 6300 – 03
TABLE A2.2 Critical 1 % Values of Cochran’s Criterion for n Variance Estimates and y Degrees of FreedomA
Degrees of Freedom y
n 1 2 3 4 5 10 15 20 30 50
3 0.9933 0.9423 0.8831 0.8335 0.7933 0.6743 0.6145 0.5775 0.5327 0.4872
4 0.9676 0.8643 0.7814 0.7212 0.6761 0.5536 0.4964 0.4620 0.4213 0.3808
5 0.9279 0.7885 0.6957 0.6329 0.5875 0.4697 0.4168 0.3855 0.3489 0.3131
6 0.8828 0.7218 0.6258 0.5635 0.5195 0.4084 0.3597 0.3312 0.2982 0.2661
7 0.8376 0.6644 0.5685 0.5080 0.4659 0.3616 0.3167 0.2907 0.2606 0.2316
8 0.7945 0.6152 0.5209 0.4627 0.4227 0.3248 0.2832 0.2592 0.2316 0.2052
9 0.7544 0.5727 0.4810 0.4251 0.3870 0.2950 0.2563 0.2340 0.2086 0.1842
10 0.7175 0.5358 0.4469 0.3934 0.3572 0.2704 0.2342 0.2135 0.1898 0.1673
11 0.6837 0.5036 0.4175 0.3663 0.3318 0.2497 0.2157 0.1963 0.1742 0.1532
12 0.6528 0.4751 0.3919 0.3428 0.3099 0.2321 0.2000 0.1818 0.1611 0.1414
13 0.6245 0.4498 0.3695 0.3223 0.2909 0.2169 0.1865 0.1693 0.1498 0.1313
14 0.5985 0.4272 0.3495 0.3043 0.2741 0.2036 0.1748 0.1585 0.1400 0.1226
15 0.5747 0.4069 0.3318 0.2882 0.2593 0.1919 0.1645 0.1490 0.1315 0.1150
20 0.4799 0.3297 0.2654 0.2288 0.2048 0.1496 0.1274 0.1150 0.1010 0.0879
25 0.4130 0.2782 0.2220 0.1904 0.1699 0.1230 0.1043 0.0939 0.0822 0.0713
30 0.3632 0.2412 0.1914 0.1635 0.1455 0.1046 0.0885 0.0794 0.0694 0.0600
35 0.3247 0.2134 0.1685 0.1435 0.1274 0.0912 0.0769 0.0690 0.0601 0.0519
40 0.2940 0.1916 0.1507 0.1281 0.1136 0.0809 0.0681 0.0610 0.0531 0.0457
45 0.2690 0.1740 0.1364 0.1158 0.1025 0.0727 0.0611 0.0547 0.0475 0.0409
50 0.2481 0.1596 0.1248 0.1057 0.0935 0.0661 0.0555 0.0496 0.0431 0.0370
60 0.2151 0.1371 0.1068 0.0902 0.0796 0.0561 0.0469 0.0419 0.0363 0.0311
70 0.1903 0.1204 0.0935 0.0788 0.0695 0.0487 0.0407 0.0363 0.0314 0.0269
80 0.1709 0.1075 0.0832 0.0701 0.0617 0.0431 0.0360 0.0320 0.0277 0.0236
90 0.1553 0.0972 0.0751 0.0631 0.0555 0.0387 0.0322 0.0287 0.0248 0.0211
100 0.1424 0.0888 0.0685 0.0575 0.0505 0.0351 0.0292 0.0260 0.0224 0.0191
A
These values are slightly conservative approximations calculated via Bonferroni’s inequality (3) as the upper 0.01/n fractile of the beta distribution. If intermediate
values are required along the n-axis, they may be obtained by linear interpolation of the reciprocals of the tabulated values. If intermediate values are required along the
v-axis, they may be obtained by second order interpolation of the reciprocals of the tabulated values.

A2.7.1 Critical values Z corresponding to a single-sided and where µ and s are the mean and standard deviation
probability P, or to a double-sided significance level 2 (1–P) respectively of the normal distribution.
are given below in terms of the “standard normal deviate,”
where
x–µ
Z5 s (A2.4)

19
 D 6300 – 03
TABLE A2.3 Critical Values of t
Double-Sided % Significance Level
Degrees of Freedom
50 40 30 20 10 5 1
1 1.000 1.376 1.963 3.078 6.314 12.706 63.657
2 0.816 1.061 1.386 1.886 2.920 4.303 9.925
3 0.765 0.978 1.250 1.638 2.353 3.182 5.841
4 0.741 0.941 1.190 1.533 2.132 2.776 4.604
5 0.727 0.920 1.156 1.476 2.015 2.571 4.032
6 0.718 0.906 1.134 1.440 1.943 2.447 3.707
7 0.711 0.896 1.119 1.415 1.895 2.365 3.499
8 0.706 0.889 1.108 1.397 1.860 2.306 3.355
9 0.703 0.883 1.100 1.383 1.833 2.262 3.250
10 0.700 0.879 1.093 1.372 1.812 2.228 3.165
11 0.697 0.876 1.088 1.363 1.796 2.201 3.106
12 0.695 0.873 1.083 1.356 1.782 2.179 3.055
13 0.694 0.870 1.079 1.350 1.771 2.160 3.012
14 0.692 0.868 1.076 1.345 1.761 2.145 2.977
15 0.691 0.866 1.074 1.341 1.753 2.131 2.947
16 0.690 0.865 1.071 1.337 1.746 2.120 2.921
17 0.689 0.863 1.069 1.333 1.740 2.110 2.898
18 0.688 0.862 1.067 1.330 1.734 2.101 2.878
19 0.688 0.861 1.066 1.328 1.729 2.093 2.861
20 0.687 0.860 1.064 1.325 1.725 2.086 2.845
21 0.686 0.859 1.063 1.323 1.721 2.080 2.831
22 0.686 0.858 1.061 1.321 1.717 2.074 2.819
23 0.685 0.858 1.060 1.319 1.714 2.069 2.807
24 0.685 0.857 1.059 1.318 1.711 2.064 2.797
25 0.684 0.856 1.058 1.316 1.708 2.060 2.787
26 0.684 0.856 1.058 1.315 1.706 2.056 2.779
27 0.684 0.855 1.057 1.314 1.703 2.052 2.771
28 0.683 0.855 1.056 1.313 1.701 2.048 2.763
29 0.683 0.854 1.055 1.311 1.699 2.045 2.756
30 0.683 0.854 1.055 1.310 1.697 2.042 2.750
40 0.681 0.851 1.050 1.303 1.684 2.021 2.704
50 0.680 0.849 1.048 1.299 1.676 2.008 2.678
60 0.679 0.848 1.046 1.296 1.671 2.000 2.660
120 0.677 0.845 1.041 1.289 1.658 1.980 2.617
` 0.674 0.842 1.036 1.282 1.645 1.960 2.576

20
 D 6300 – 03
TABLE A2.4 Constants for Approximating Critical Values of FA
100 (1–P) % A(P) B(P) C(P)
10.0 % 1.1131 0.77 0.527
5.0 % 1.4287 0.95 0.681
2.5 % 1.7023 1.14 0.846
1.0 % 2.0206 1.40 1.073
0.5 % 2.2373 1.61 1.250
0.1 % 2.6841 2.09 1.672
0.05 % 2.8580 2.30 1.857
A
For values of P not given above, critical values of F may be obtained by
second order interpolation/extrapolation of log (F) (either tabulated or estimated
from the formula) against log (1–P).

TABLE A2.5 Critical Values of the Normal DistributionA

P 0.70 0.80 0.90 0.95 0.975 0.99 0.995
Z 0.524 0.842 1.282 1.645 1.960 2.326 2.576
2(1–P) 0.60 0.40 0.20 0.10 0.05 0.02 0.01
A
When P is less than 0.5 the appropriate critical value is the negative of the
value corresponding to a probability (1–P).

A3. TYPES OF DEPENDENCE AND CORRESPONDING TRANSFORMATIONS (7.2)

A3.1 Types of Dependence of scatter diagrams. Refer to Figs. A3.1-A3.6 and identify the
A3.1.1 See Table A3.1. type of transformation to be applied (if any).
A3.2.1.2 With the exception of the power transformation
A3.2 Transformation Procedure (Type 2 in Table A3.1), the transformation parameter is either
A3.2.1 The following steps shall be taken in identifying the known in advance or estimated from the scatter diagrams. For
correct type of transformation and its parameters, B or B0, or the arcsin (Type 3) and logistic (Type 4) transformations, B will
both. be the upper limit of the rating scale or “score” that defines
A3.2.1.1 Plot laboratories standard deviations, D, and re- results. For the log (Type 1) transformation, calculate B0 from
peats standard deviations, d, against sample means in the form the intercept and slope (B0 = intercept/slope), estimated from

TABLE A3.1 Types of DependenceA

Form of Dependence Transformations Form of Line to be Fitted dx/dy Remarks
D = K(m + B0) y = log(x + B0) log(D) = bo+ (x + B0) Care must be taken if (x + B0) is small, as
m + B0> 0 Type 1 – “log” +b1log(m + B0) + b2T + b3Tlog(m + B0) rounding becomes critical

Test: b1 = 1, b3= 0

D = K(m+B0)B y =( x+B0)1–B log(D) = bo+ Blog(m + B0)+ b2T + (x + B0)B/(1 - B) B = 1⁄2 or 2 are common cases.
m + B0> 0, Type 2 – “power” b3Tlog(m+B0) If B is not different from 1, use log
B fi 1 Test: B fi 1, b3= 0 transform 1 above. The fitted line may pass
through the origin.

D=K[(m/B) (1-m/B)]1/2 y=arcsin(x/B)1/2 log(D) = bo+ b1log[m (B-m)] + b2T + 2[x (B-x)]1/2 This case often arises when results are
b3Tlog[m (B – m)] reported as percentages or qualitatively as
0#m#B Type 3 – “arcsin” “scores.” If x is always small compared to
Test: b1= 1/2, b3= 0 B, the transformation reduces to y=(x)1/2, a
special case of 2 above.

D=K[ (m/B)(1-m/B)] y=log[x/(B-x)] log(D)= bo+ b1log[m (B-m)] + b2T + x (B-x)/B This case arises when results are reported
b3Tlog[m (B – m)] on a scale of 0 to B. If x is always small
0#m#B Type 4 – “logistic” compared to B, then the transformation
reduces to y = log(x) a special case of 1
Test: b1 = 1, b3= 0 above.

D=K[(m2+B2)/B] y = arctan(x/B) log(D)= bo+ b1log(m2+B2) + b2T + (x2+ B2)/B The fitted line does not pass through the
b3Tlog(m2+B2) origin. If B is small, the transformation
B>0 Type 5 – “arctan” reduces to y = 1/x, a special case of 2
Test: b1 = 1, b3= 0 above.
A
The forms of dependence above are shown graphically in the corresponding Figs. A3.1-A3.6. In all cases, K can be any positive constant, and “log” refers to natural
logarithms. The form of line to be fitted includes a dummy variable T (see A4.1) by which it is possible to test for a difference in the transformation as applied to repeatability
and reproducibility.

21
 D 6300 – 03

FIG. A3.1 Type 1, log FIG. A3.3 Type 2, power

FIG. A3.2 Type 2, power

FIG. A3.4 Type 3, arcsin
the scatter diagrams. Similarly, estimate B from the intercept in
the case of the arctan (Type 5) transformation. In every case, B
or B0, or both, shall be rounded to give a meaningful value that ing to the transformation in question, in accordance with the
satisfies the plots for both the laboratories and repeats standard computational procedure in A4.3. For the power transforma-
deviations. tion, coefficient B, shall differ significantly from zero and shall
A3.2.1.3 In the case of the power transform, B and B0 = 0 be rounded to a meaningful value. For the arcsin transforma-
will be estimated as part of the line fitting procedure described tion, b1 shall have a value not significantly different from 0.5.
in the next section (A3.2.1.4). A non-zero B0 may be estimated Similarly, b1 shall not significantly differ from a value of one
by minimizing the sum of squared residuals from the fitted line. for the logistic, log, and arctan transformations. In every case
Function minimization using a simplex procedure due to the test specified in Table A3.1 shall be applied at the 5 %
Nelder and Meade (9,10) has been found satisfactory. This is significance level. Failure of this test implies either that the
applied to the functional form of the line shown in Table A3.1 type of transformation or its parameter B is incorrect. Similarly,
using the calculated sample means and standard deviations. coefficient b3 shall in every case be tested as zero. Failure in
The values and significances of all the constants are determined this case implies that the transformation is different for
simultaneously as part of the simplex minimization. For repeatability and reproducibility. In some cases the presence of
detailed discussion of simplex minimization consult a trained outliers (see 7.3) can give rise to this difference.
statistician. A3.2.1.5 If the tests applied above were satisfactory, trans-
A3.2.1.4 In order to confirm the selected transformation form all the results accordingly, recalculate means and standard
type, and to estimate the parameter B in the case of the power deviations using transformed results, and create new scatter
transformation, fit the line specified in Table A3.1, correspond- diagrams as in A3.2.1. These will now show a uniform level for

22
 D 6300 – 03

FIG. A3.6 Type 5, arctan

FIG. A3.5 Type 4, logistic

sarily the same) level for repeats standard deviation. A statis-

laboratories standard deviation, and a uniform (but not neces- tical test for uniformity is given in 7.4.

A4. WEIGHTED LINEAR REGRESSION ANALYSIS (7.2)

A4.1 Explanation for Use of a Dummy Variable estimation of precision relationships. An “importance ratio” of
A4.1.1 Two different variables Y1 and Y2, when plotted 2:1 in the favor of reproducibility shall be applied by setting T1
against the same independent variable X, will in general give = 1 and T2 = –2, where T 1 refers to the plot of laboratories
different linear relationships of the form standard deviation and T2 refers to the repeats standard devia-
tion.
Y1 5 b10 1 b11X (A4.1)
Y2 5 b20 1 b21X A4.2 Derivation of Weights Used in Regression Analysis
where the coefficients bij are estimated by regression analy- A4.2.1 In order to account for the relative precision of fitted
sis. In order to compare the two relationships, a dummy variables in a regression analysis, weights shall be used that are
variable T can be defined such that inversely proportional to the variances of the fitted variables.
T = T1, a constant value for every observation of Y1, A4.2.1.1 For a variable D, which is an estimate of popula-
T = T2, a constant value for every observation of Y2, and tion standard deviation s, based on v (D) degrees of freedom,
T1fi T2
the variance of D is given by
A4.1.2 Letting Y represent the combination of Y1 and Y2, Var ~D! 5 s2/2v ~D! (A4.6)
plot a single relationship
A4.2.1.2 Replacing s by its estimate D , the weight for this
2 2
Y 5 b0 1 b1X 1 b2T 1 b3TX (A4.2)
variable will be approximated by
where, as before, the coefficients bi are estimated by regres-
w~D! 5 2v ~D! / D2 (A4.7)
sion analysis. By comparing Eq A4.1 and Eq A4.2), it is
evident that A4.2.1.3 It is clear that as standard deviation D increases, so
will the weight decrease. For this reason the fitted variable in
b10 5 b0 1 b2T1 (A4.3)
the weighted regression shall instead be a function of standard
b20 5 b0 1 b2T2 deviation, which yields weights independent of the fitted
and that therefore variable.
b10 – b20 5 b2 ~T1 – T2! (A4.4) A4.2.1.4 In cases where a function g(D) is fitted, rather than
D itself, the variance formula becomes
A4.1.3 Similarly,
1 1 s2
b11 – b21 5 b3 ~T1 – T2! (A4.5) Var @ log ~D!# 5 2 Var ~D ! 5 2 2v ~D ! (A4.8)
D D
A4.1.4 In order to test for a difference between b10 and b20
A4.2.1.5 Once again replacing s2 by its estimate D2, the
therefore, it is only necessary to test for a non-zero coefficient
weight for log(D) will be approximated by
b2. Similarly, to test for a difference between b11 and b21, test
for a non-zero coefficient b3. w@log ~D!# 5 2v ~D! (A4.9)
A4.1.5 Any non-zero values can be chosen for T1 and T2. A4.2.1.6 In relation to laboratories standard deviation D and
However, since reproducibility is the basis of tests for quality repeats standard deviation d, therefore, it is necessary to
control against specifications, weighting shall reflect this in the perform regression analysis in terms of log(D) and log(d),

23
 D 6300 – 03
since weighting will then take account only of the amount of ay1 5 a11b1 1 a12b2 1 a13b3 (A4.15)
data on which the standard deviation was based. A relationship ay2 5 a21b1 1 a22b2 1 a23b3
estimated in this way will be less dependent on samples which ay3 5 a31b1 1 a32b2 1 a33b 3
have a high proportion of missing results.
A4.2.1.7 Denoting degrees of freedom as v(D) for labora- A4.3.1.5 Examples of aij and ayi elements, in terms of
tory standard deviations D and v(d) for repeats standard weighted means x̄i, are as follows
deviations d, formulae for calculating weights then become a22 5 (wi ~x2i – x̄2! 2 a23 5 (wi ~x2i – x̄2! ~x3i – x̄3!
w@log ~D!# 5 2v ~D! (A4.10) (A4.16)
2
ay2 5 (wi ~yi – ȳ! ~x2i – x̄2! ayy 5 (wi ~yi – ȳ!
w@log ~d!# 5 2v ~d! (A4.11) A4.3.1.6 Having solved the equations for b1, b2, and b3,
NOTE A4.1—Unweighted regression corresponds to weighted regres- calculate the intercept from the weighted means of the vari-
sion in which all the weights have a constant value 1. ables as
b0 5 ȳ – b1x̄1 – b2x̄2 – b3x̄3 (A4.17)
A4.3 Computational Procedure for Regression Analysis
A4.3.1.7 Coefficient estimates bi can be summarized in
A4.3.1 The following technique gives the best fitting
tabular form, together with test statistics, as in Table A4.2.
straight line of the form of Eq A4.2.
A4.3.1.8 In order to complete the table, it is necessary to
A4.3.1.1 First draw up a table (see Table A4.1) giving
calculate the standard deviation of the observed y values about
values of the variables to be plotted in the regression, together
the estimated line. This is called the residual standard devia-
with corresponding weights. Functions g1 and g2 will always be
tion, and is given by
natural logarithms corresponding to the transformation in
question, as specified in A3.2.
A4.3.1.2 Using the symbols defined in Table A4.1, the line
s5 Π1
n – 4 ~ayy – b1ay1 – b2ay2 – b3ay3! (A4.18)

to be fitted (Eq A4.2) becomes A4.3.1.9 Standard errors of the estimates then become
y 5 b0 1 b1x1 1 b2x 2 1 b3x3 (A4.12) ei 5 s=cii, for i 5 1 to 3 (A4.19)
A4.3.1.3 The intercept b0 can be eliminated by rewriting and
this as
e0 5
~y – ȳ! 5 b1 ~x1 –x̄1! 1 b2 ~x2 – x̄2! 1 b3 ~x3 – x̄3!
where y, x1, x2, and x3 are weighted mean values, for
(A4.13)
sΠ1 2 2 2
n 1 c11x̄1 1 c22x̄2 1 c33x̄3 1 2c12x̄1x̄2 1 2c13x̄1x̄3 1 2c23x̄2x̄3
example (A4.20)
n where the elements cjj correspond to the inverse of the matrix
( wix2i
i21
containing elements ajj.
x̄2 5 n (A4.14) A4.3.1.10 The t-ratios are the ratios (bi–K)/ ej, where K is a
( wi
i51
constant, and by comparing these to the critical values of t in
Table A2.3, it is possible to test if coefficient bi differs from K.
and where n is the number of points (twice the number of If ti is greater than the critical value corresponding to 5 %
samples) to be plotted. significance and (n–4) degrees of freedom, then the coefficient
A4.3.1.4 The least squares solution of Eq A4.14 requires the can be regarded as differing from K. In particular, t1 will
solution of the set of simultaneous equations of the form identify an inappropriate slope b1 and t3 will indicate whether
the slope is different for laboratories and repeats standard
TABLE A4.1 Arrangement of Variables for Regression Analysis deviations. Since laboratories standard deviation will generally
Standard
Sample Mean be larger than repeats standard deviation at the same level of
Deviation
Sample
Function
Function Dummy T Tg2 Weight sample mean, t2 will in general indicate a non-zero coefficient
g2 b2 .
g1
1 g1 (D1) g2 (m1) 1 g2 (m1) 2y (D1)
2 g1 (D2) g2 (m2) 1 g2 (m2) 2y (D2) A4.4 Worked Example
3 g1 (D3) g2 (m3) 1 g2 (m3) 2y (D3)
· · · · · ·
A4.4.1 This section describes the fitting of a power function
· · · · · · (Type 2 of Table A3.1) using weighted linear regression
· · · · · · according to the procedure of A3.2. Rounded sample means
S g1 (Ds) g2 (ms) 1 g2 (ms) 2y (Ds)

1 g1 (d1) g2 (m1) –2 –2g2(m1) 2y (d1)

2 g1 (d2) g2 (m2) –2 –2g2(m2) 2y (d2) TABLE A4.2 Presentation of Estimates from Regression Analysis
3 g1 (d3) g2 (m3) –2 –2g2(m3) 2y (d3)
Fitted Coefficient Standard Error of
· · · · · · t-Ratio
Variable Estimate Estimate
· · · · · ·
· · · · · · Intercept b0 e0 t0
S g1 (ds) g2 (ms) –2 –2g2(ms) 2y (ds) Sample Mean b1 e1 t1
Dummy b2 e2 t2
Symbol yi x1j x2i x3i wi Dummy 3 mean b3 e3 t3

24
 D 6300 – 03
and standard deviations are given in Table 3, 7.2, based on the TABLE A4.3 Arrangement of Variables for Sample Data
bromine number data in A2.1. Logarithm of
Logarithm of Dummy 3 log
A4.4.1.1 Scatter diagrams identified the power transforma- Sample Standard Dummy T Weight
Sample Mean (mean)
Deviation
tion as appropriate, as indicated by the log-log plot shown in
1 –0.3158 0.7655 1 0.7655 16
Fig. A4.1. 2 0.7969 4.1804 1 4.1804 18
A4.4.1.2 Transformation parameter B need not be estimated 3 –2.7046 –0.2802 1 –0.2802 28
from Fig. A4.1, since it will be given in the regression analysis 4 –1.5568 1.2932 1 1.2932 22
5 –1.2358 2.3888 1 2.3888 18
that follows. 6 0.4029 3.8755 1 3.8755 18
A4.4.1.3 The form of the line to be fitted (Table A3.1) is 7 1.0762 4.7378 1 4.7378 18
8 –1.8401 0.1975 1 0.1975 18
log~D! 5 b0 1 b1log ~m! 1 b2T 1 b3Tlog ~m! (A4.21)
A4.4.1.4 The table of values to be fitted (see Table A4.1) is 1 –2.0644 0.7655 –2 –1.5309 18
2 –0.2015 4.1804 –2 –8.3609 18
shown in Table A4.3. 3 –2.9957 –0.2802 –2 0.5605 18
A4.4.1.5 Least squares regression requires the solution of 4 –2.1585 1.2932 –2 –2.5864 18
the simultaneous equations 5 –2.3613 2.3888 –2 −4.7775 18
6 –0.6415 3.8755 –2 –7.7510 18
614.671 5 999.894b1 – 35.8524b2 – 493.045b3 (A4.22) 7 −0.0674 4.7378 –2 −9.4756 18
8 –2.8612 0.1975 –2 −0.3949 18
188.526 5 35.8524b1 1 673.920b2 1 1409.58b3
195.477 5 –493.045b1 1 1409.58b2 1 5362.27b3 Symbol yi x1i x2i x3i wi

A4.4.1.6 Also required are

ayy 5 505.668 (A4.23) TABLE A4.4 Presentation of Estimates from Sample Data
s 5 2.23868 Fitted Variable
Coefficient Estimate Standard Error of
t-Ratio
bi Estimate
A4.4.1.7 The solution is summarized in Table A4.4 (see
Intercept –2.4064
Table A4.2): Log (mean) 0.63773 0.07359 8.67
A4.4.1.8 Comparing the t-ratios with the critical 5 % values Dummy 0.25496 0.13052 1.95
Dummy 3 log (mean) 0.02808 0.04731 0.59
for 12 degrees of freedom (namely 2.179) given in Table A2.3,
it can be seen that the slope is significantly non-zero (b1 =

0.638), confirming that a transformation was required. Further-

more, since coefficient b3 does not significantly differ from
zero, the slope (and resulting transformation) is the same for
both laboratories and repeats standard deviations.
A4.4.1.9 As the slope b1 = 0.638 has a standard error of
0.074, the approximate 66 % confidence region of 0.638 6
0.074 will contain the value 2/3. Rounding to this value is
therefore reasonable, and leads to the convenient transforma-
tion
y 5 x1/3 (A4.24)
A4.4.1.10 Having applied this transformation and recalcu-
lated sample means and standard deviations, corresponding
scatter diagrams are shown in Fig. A4.2. These show uniform
levels for both laboratories and repeats standard deviations for
all samples except Sample 1. In the case of the latter sample,
FIG. A4.1 Precisions Vary with Level the extreme point is due to outliers.

25
 D 6300 – 03

FIG. A4.2 After Transforming, Precisions Do Not Vary with Level

APPENDIX

(Nonmandatory Information)

X1. DERIVATION OF FORMULA FOR CALCULATING THE NUMBER OF SAMPLES REQUIRED (see 6.4.3)

X1.1 An analysis of variance is carried out on the results of Q = s2 2/s02,

the pilot program. Setting the three expressions in 8.3.1 equal v = reproducibility degrees of freedom,
to the corresponding mean squares and solving yields rough L = number of laboratories, and
estimates of the three components of variance, namely: S = number of samples.
s02 for repeats,
s12 for laboratories 3 samples interaction, and X1.3 The formula rearranges into the form
s22 for laboratories. aS 1 b 5 0 (X1.2)

X1.2 Substituting the above in Eq 39 (8.3.3.3) for calcu- where:

lating the reproducibility degrees of freedom, this becomes a = vQ2 – (1 + P + Q)2( L – 1), and
b = v[(2Q + 1/2 + P) (1/2 + P) + 0.25 (L– 1) / L].
~1 1 P 1 Q!2 @~1/2 1 P! / S 1 Q#2 ~S – 1! ~1/2 1 P!2 1 X1.3.1 Therefore S = –b/a gives the values of S for given
5 1 1 4LS
v ~L – 1! 2
S ~L – 1! values of L, P, Q, and v.
(X1.1)
X1.4 Fig. 1 is based on v = 30 degrees of freedom. For
where: non-integral values of P and Q, S can be estimated by second
P = s12/s02,
order interpolation from the table.

REFERENCES

(1) Standard Methods for Analysis and Testing of Petroleum and Related second ed., 1963, Example 6B.1, pp. 236-238.
Products, The Institute of Petroleum, London, England, 1993, Appen- (6) Kolodziejczyk, S., Biometrika, Vol 27, 1935, pp. 161-190.
dix E. (7) Welch, B. L., Biometrika, Vol 29, 1938, pp. 350-362.
(2) Shapiro, S. S., and Wilks, M. B., Biometrika, Vol 52, 1965, pp.
591-611. (8) Merrington, M., and Thompson, C. M., Biometrika, Vol 33, 1943, pp.
(3) Cochran, W. G., Ann. Eugen., Vol 11, 1941, pp. 47-52. 73-88.
(4) Hawkins, D. M., Identification of Outliers, 1980, pp. 136-138. (9) Nelder.
(5) Davies, O. L., et al, Design and Analysis of Industrial Experiments, (10) Meade.

26
 D 6300 – 03
ASTM International takes no position respecting the validity of any patent rights asserted in connection with any item mentioned
in this standard. Users of this standard are expressly advised that determination of the validity of any such patent rights, and the risk
of infringement of such rights, are entirely their own responsibility.

This standard is subject to revision at any time by the responsible technical committee and must be reviewed every five years and
if not revised, either reapproved or withdrawn. Your comments are invited either for revision of this standard or for additional standards
and should be addressed to ASTM International Headquarters. Your comments will receive careful consideration at a meeting of the
responsible technical committee, which you may attend. If you feel that your comments have not received a fair hearing you should
make your views known to the ASTM Committee on Standards, at the address shown below.

This standard is copyrighted by ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959,
United States. Individual reprints (single or multiple copies) of this standard may be obtained by contacting ASTM at the above
address or at 610-832-9585 (phone), 610-832-9555 (fax), or service@astm.org (e-mail); or through the ASTM website
(www.astm.org).

27