In the context of testing laboratories where an ISO 17025 Competence Requirements system is implemented, it is well known that, in section 7.7 on Ensuring the validity of results, it is stated: “The laboratory shall have a procedure for monitoring the validity of results. The resulting data shall be recorded in such a way that trends are detectable and, where possible, statistical techniques shall be applied for the review of results.”
A common tool for this trend evaluation is the use of control charts. The primary purpose of these charts is not to verify compliance with individual value tolerances (this can be done mathematically without a chart), but rather to assess whether, over time, the method is preserved and maintained as it was at the time of its validation, as well as to detect trends (assessing a dataset and its statistical behavior over time).
A control chart represents a Normal statistical model in which 50% of the data are expected to fall on one side and 50% on the other side of the central value. When a method has significant bias, this condition is not met, and the laboratory should act by modifying and improving the method, which is not always possible.
It is clear that, with samples of unknown value, the central value of a chart is built using the mean of a series of results, and the limits with ± 2SR and ± 3SR (warning and control, respectively). However, when a reference value is available, there is the option to work either with the theoretical value or with the mean value, which depends on whether or not the method has significant bias.
This will be verified during method validation, assessing whether there are statistically significant differences between the obtained results and the expected theoretical values.
If there are no statistically significant differences, the central value can be set as either the mean value or the reference value, as indicated by the reference standards, since in this case the data should be distributed equally on both sides of the central value, whichever it is.
If there are statistically significant differences, then the central value must be set as the mean value obtained. Otherwise, if the reference value is used, the results will not fluctuate around it but will always be above or below the central line, and the control chart would lose its purpose, which is to evaluate trends, only identifying, at most, a systematic trend that is already known.
In summary:
- Control charts serve to assess the statistical behavior of the method, ensuring that it remains stable over time.
- If validation already showed that the method has a bias, it is expected that this bias will persist over time.
- Control charts are designed to detect trends in a dataset, not to evaluate the accuracy of an individual value. Hence the need for decision-making rules. Among them is the rule that if “x” consecutive values are found on the same side of the central value (usually 8), it indicates a possible trend, typically associated with bias. If this bias was already identified during validation and is not accounted for in the chart construction (i.e., if the central value is set as the theoretical value instead of the mean obtained), the data will appear shifted to one side. This prevents the evaluation of variations around the already-identified bias (which should remain stable over time), and the interpretation rules lose their meaning.
The above is based not only on basic general concepts of statistics but also on well-recognized and widely accepted publications in the field, as listed below:
- ISO 7870-2:2013
The reference standard for Shewhart control charts, ISO 7870-2, states that among the types of charts that can be used are those without pre-specified values and those with them. In section 5.2, which explains the basis of the latter, it is indicated that, preferably, the specified values should come from a preliminary investigation of data, which is assumed to represent the typical behavior of future data, considering the inherent variability of the process. Therefore, in our context, this refers to method validation studies (it refers to “pre-defined values,” not “reference values”).
- Standard Methods
In Standard Methods, reference is made to control charts, indicating that they represent a graphical record of quality, placing controls over time to demonstrate the statistical control of an analytical process and to detect possible changes that may arise. It also specifies that accuracy charts (means) are constructed using the mean value, clarifying that the analyzed samples may include, among others, calibration standards (thus, for samples with a reference value, it also indicates construction with mean values). A graphical example is included showing that the central value is not based on the expected theoretical values but on the means obtained in prior studies.
- AEAS Guide Part I (Rev.1)
This AEAS guide indicates that acceptance criteria must be related to the results declared during validation, suggesting that initial control charts should be built from validation data.
In Annex 3, with an example of a chart using a reference material, it is stated that since there are no significant differences between the experimental mean value and the reference value of the material in use, the central line in the control chart can be set either from the mean of the control values or as the reference value itself. Therefore, in the opposite case (where such significant differences exist), the choice of the central value becomes relevant, and due to these differences, it makes sense to use the mean values.
Furthermore, in Annex V, there is a direct example where, despite having a reference value, the central value is set as the mean (microbiological method):
Conclusion: In methods where statistically significant differences exist between the expected and obtained results, the central value in a control chart should be the obtained mean value rather than the reference value.
