Food Standards Agency
Thursday 2 July 2009
Safer food better business banner
AZ-Directory
What's NewRSS
What is RSS?Listen
Listen to this siteThursday 7 October 2004
This research project aims to investigate the modification of a standard ruggedness test to provide data with the same validity as a collaborative trial, but at a reduced cost.
Study Duration : January 2001 to April 2002
Contractor : Birkbeck College, University of London
Chemical measurements, such as the concentration of a contaminant in a food, underlie most decisions about whether a particular foodstuff is fit to eat. All measurements, including chemical measurements, contain 'uncertainty', a term that indicates the range in which we believe the true result to lie with a high probability. The uncertainty associated with a result is an essential aid to making decisions based on the result gained.
The collaborative trial is one means of providing a benchmark uncertainty for the result of a chemical measurement. However, these trials involve work by a large number of laboratories and are therefore expensive, typically costing £30,000 per analytical method. The 'ruggedness test' is another possible approach to estimating uncertainty. It is carried out in one laboratory, and therefore much cheaper.
Analytical methods comprise a list of instructions for the chemical operations needed to make the measurement, rather like a recipe. In a ruggedness test, small variations are deliberately introduced into the recipe to see how it affects the outcome. In a good, or 'rugged' method, the measurement result is unaffected by these minor variations. The idea underlying this research is that by making the variations to the recipe appropriately larger, it would be possible to simulate the degree of variation that would be observed in a collaborative trial. However, the standard ruggedness test is only regarded as a screening test and incapable of providing good estimates of uncertainty.
The aim of the research was to determine whether a standard ruggedness test could be modified in such a way as to provide data with the same validity as a collaborative trial but at a much smaller cost.
In this study ruggedness tests, suitably designed by a panel of four experts, are applied to ten different chemical analyses. The tests selected are on a range of constituents of foodstuffs (trace elements, trace organic compounds, anions, proximate analytes), covering a concentration range from low traces to major component (89ppb to 56%) and with a variety of chemical and physical principles underlying the procedure. The methods used were selected on the basis of there being a number of published collaborative trials to provide comparison data. All tests being executed by scientists experienced in the methodologies being used.
Critical features of each analytical method are considered and determined in advance by the panel of experts, with the perturbations in the method features being determined by consideration of the uncontrolled variation thought likely to occur between laboratories. However, no factors are considered in the experimental design to accommodate excursions from the written procedure, other than those determined in the design of the ruggedness experiment. The combination of perturbed levels for each complete analysis are to be determined at random, and 20 such complete analyses undertaken for each method on a homogenised laboratory sample of the relevant test material.
After piloting the test on an analytical technique, and a review of the initial design, the modified ruggedness test to will be applied to four more analytical methods in the food sector. After further adjustments to the design, tests will be carried out on the remaining five methods, to compare the uncertainty estimate from the ruggedness tests with estimated as reproducibility precision in the collaborative trial.
This pilot study has shown that in a series of ruggedness tests it was not possible to simulate reproducibility standard deviation by means of carefully considered perturbations of the analytical procedure in a Single Laboratory, with the possible exception of those methods not involving an empirical calibration. The estimated standard deviations derived from the experiments are too small, by a factor of about two, and are therefore close to repeatability standard deviations. Given this outcome, this study provides no grounds for extending the ruggedness study to a wider range of analytical systems.
In three out of the ten types of measurement studied, the modified ruggedness test provided an estimate of uncertainty similar to that given by the collaborative trial covering the same analytical method. These were all methods that did not require an empirical calibration in the procedure, i.e. they only involved measurements of mass and volume to determine the concentration of the analyte. In the other seven types of measurement (all involving an empirical calibration) the ruggedness test provided an estimate of uncertainty that was lower than that likely to be furnished by a collaborative trial, by 46%. In fact, the estimated uncertainties were very similar to those normally estimated by within-laboratory operations.
This suggests that it seems not to be possible to save money on method validation and uncertainly estimation by replacing collaborative trials with ruggedness tests. Moreover it acts as a warning that a validation of an analytical method that does not involve an inter-laboratory exercise is likely to provide a serious underestimate of uncertainty.
Thompson, M., Guffogg, S., Stangroom, S., Osborne, P., Key, P. and Wood, R. (2002). Precision estimates produced by specially designed ruggedness tests compared with those derived from collaborative trials, in relation to estimation of measurement uncertainty. Analyst, 127, 1669-1675.
The final report is available from the FSA Library and Information centre.
To obtain a copy, please contact the Enquiry Desk, Dr Elsie Widdowson Library and Information Services, Food Standards Agency (tel: 020 7276 8181/8182 or email:
library&info@foodstandards.gsi.gov.uk
)
Contact
: For any enquiries concerning this research project, please contact the relevant Programme contact or email
science@foodstandards.gsi.gov.uk
Find out what our other sites have to offer