Larry's Curb Your Anticipation, Part 5:Test Development and Validation

Up to the Introduction; Back to part 4; On to part 6.

By Commenter Larry

Let’s get started looking at the ISO laboratory rules that apply, finishing here with "Fitness for Purpose".

Rules Governing Test Method Development and Validation

Our first category, test method and development, is probably our most complicated category. It is one of the most detailed categories under ISO 17025, and the longest single section in the ISL. Arguably, this is also the most important single category applicable to anti-doping testing. Why should this be the case?

[MORE]

The answer is set forth in ISL Rule 5.4.4.1:

Standard methods are generally not available for Doping Control analyses. The Laboratory shall develop, validate, and document in-house methods for compounds present on the Prohibited List and for related substances. The methods shall be selected and validated so they are fit for the purpose.

ISL Rule 5.4.4.1 points out a critically important feature of anti-doping testing: standard methods are generally not available for Doping Control analyses. Consequently, the ISL requires each WADA lab to come up with its own tests “in house”. Given the importance of these tests, it makes sense that a substantial portion of the ISL addresses the process of developing and validating these tests.

We can divide our discussion under this heading into two parts: method development and method validation. We will examine each of these concepts in turn. In the process, we’ll look at a number of the terms we’ve used in our discussions here at TBV: terms such as “specificity”, and “traceability”, and “uncertainty”. I will do my best to define these terms, but you should note that these terms can have different meanings in different contexts, and that some of these terms do not have universally accepted definitions. I will try to define these terms in a way I think is consistent with the ISL and ISO 17025, and I will always provide a citation to support the definition I have chosen, but you may well be able to find alternative definitions for some of these terms.

1. Test Method Development

ISO 17025 assumes that a lab will select existing methods whenever possible, so the ISO 17025 standard has comparatively little to say about method development. ISO 17025 Rule 5.4.4 contains general requirements for new test procedures: each new test procedure must set forth an appropriate scope, parameters, pre-test checks, a method of recording observations, and interestingly, “criteria and/or requirements for approval/rejection.” (What does ISO 17025 Rule 5.4.4 mean when it refers to “approval/rejection”? Does this rule require that a test method contain criteria for when the results of a test have been performed correctly and can be approved, and when the results are invalid and must be rejected? ISO 17025 does not say. What criteria did LNDD use for “approval/rejection” under ISO 17025 Rule 5.4.4? It would be interesting to know the answers to these questions.)

While the ISL has rules addressed to “Selection of Methods”, see ISL Rule 5.4.4.1, I think that these rules fit better into our discussion of Test Method Validation below.

2. Test Method Validation

Method validation rules are arguably the most important rules in the ISL. I will argue that most of the lab testing criteria we discuss here on TBV are method validation rules. However, “method validation” has a specific meaning in the world of lab testing. Thanks to Russ, we have a very good discussion of method validation available to us in the form of a Eurachem Guide on “Fitness for Purpose”. See http://www.eurachem.org/guides/valid.pdf. I will rely heavily on this Eurachem Guide in the analysis that follows.

ISO 17025 defines “method validation” as “the confirmation by examination and the provision of objective evidence that the particular requirements for a specific intended use are fulfilled.” (ISO 17025 Rule 5.4.5.1.) A shorter definition of method validation, more to the point, is “the process of verifying that a method is fit for purpose, i.e. for use for solving a particular analytical problem.” (See Eurachem Guide Annex A paragraph A33.2)

Method validation is closely connected with method development: according to the Eurachem guide, it is often not possible to determine exactly where method development finishes and validation begins. (Eurachem Guide 3.1) However, method validation is an ongoing process that does not end after the method has been developed. According to paragraph 5.1 of the Eurachem Guide:

A method should be validated when it is necessary to verify that its performance parameters are adequate for use for a particular analytical problem. For example:

• new method developed for particular problem;

• established method revised to incorporate improvements or extended to a new problem;

• when quality control indicates an established method is changing with time;

• established method used in a different laboratory, or with different analysts or different instrumentation;

• to demonstrate the equivalence between two methods, e.g. a new method and a standard.

In other words, method validation should take place not only as a new method is adopted, but also at any time the method is changed or is employed in a different way, or where the lab’s quality control efforts indicate that the method should be re-examined.

Central to method validation is the concept of “fitness for purpose” – the determination that the method under consideration has performance capabilities consistent with what the application requires. If a method is “fit for purpose”, then it is trustworthy – it provides the “right answer to the analytical part of the problem.” (See Eurachem Guide Section 4.3) So, implicit in method validation is a determination by the lab that the method is good enough, that it is worth performing, that on an absolute basis the test will meet its designed purpose.

Up to the Introduction; Back to part 4; On to part 6.