Larry's Curb Your Anticipation, Part 6: On Fitness For Purpose

Up to the Introduction; back to part 5; on to part 7

By Commenter Larry

Fitness For Purpose

As should be obvious, in order to determine whether a lab method is “fit for purpose”, the lab must determine the method’s intended purpose, or in the words of the Eurachem Guide, “the performance requirements that a method must have to be suitable for solving the analytical problem.” The process of method validation can be understood as a cycle of method development and evaluation that continues until the method is found to be capable of meeting the defined performance requirement. See Eurachem Guide paragraph 6.9.

[MORE]

ISO 17025 is written for laboratories doing work for clients, and as you would expect, ISO 17025 is primarily addressed to the responsibilities that labs must assume when they do work for clients. However, when it comes to the “fitness for purpose” standard, the client bears some responsibility for defining the purpose of the method to be selected or developed by the lab. It is up to the client in the first instance to define the requirements of the method they want the lab to perform. As stated in the Eurachem Guide, “ideally the laboratory should first agree with the customer [on] an analytical requirement which defines the performance requirements that a method must have to be suitable for solving the analytical problem.” See Eurachem Guide paragraph 6.9. Of course, most lab customers are not sophisticated enough to come up with any such definition. The Eurachem Guide acknowledges that most customers “define their requirements in terms of cost and/or time and rarely know how well methods need to perform.” Eurachem Guide paragraph 6.10.

But WADA is not a typical lab customer – WADA is a leading authority in the field of doping control, and it contracts with numbers of labs world-wide to perform doping tests. Consequently, WADA bears a significant responsibility for defining the purpose of these tests, and WADA has at least made an effort in the ISL to meet this responsibility. Under the ISL, each test developed by a WADA lab must be capable of measuring a specified minimum amount of a particular prohibited substance. This specified minimum amount is sometimes called a “limit of detection”. See ISL Rule 5.4.4.1.3. (For a discussion of “limit of detection”, see Eurachem Guide paragraph 6.20.) If the test is to detect a “non-threshold substance” (a substance that can be the basis for an AAF if found in the athlete’s system in any amount), then the test must be able to identify the substance at the limit of detection. See ISL Rule 5.4.4.1.1. If the test is supposed to detect a “threshold substance” (a substance that must be present in an athlete’s system in more than a specified amount in order for the lab to find an AAF), then the “purpose” of the test is more demanding: the test must be capable of identifying both the substance and the amount of the substance with an acceptable “uncertainty”. See ISL Rule 5.4.4.1.2. (The issue of “uncertainty” is important, and we’ll address it shortly.)

(Here at TBV, we’ve debated the fitness of methods in use at WADA labs, but we have not considered as carefully the stated purpose for these methods. Assuming for the moment that there are deficiencies in the way that these labs perform their doping tests, we should consider whether the fault may lie in part with WADA’s failure to articulate a clear and definitive purpose for these tests.)

Once a lab determines the purpose of a particular method and develops a method to meet this purpose, the lab can then consider whether the method is “fit” for this purpose. This analysis is not limited to looking at the method in a vacuum. The determination of “fitness for purpose” turns on how the method performs when used at a particular lab, by the analysts employed at that lab, using the equipment and facilities available at that lab. See Eurachem Guide paragraph 6.9. A method that might work perfectly well at a state-of-the-art lab might not be “fit for purpose” at a less sophisticated lab.

The effort involved in method validation will vary. ISO 17025 rule 5.4.5.2 states vaguely that “[t]he validation shall be as extensive as is necessary to meet the needs of the given application.” The Eurachem Guide is a bit more helpful. The Eurachem Guide states that “[c]haracterisation of method performance is an expensive process and inevitably it may be constrained by time and cost considerations,” and that the lab must strike “the balance between time and costs constraints and the need to validate the method.” How does the lab manage to strike such a critical balance? The Eurachem Guide states only that “the laboratory should do the best it can within the constraints imposed.” (Eurachem Guide paragraphs 6.6 and 6.7.) So, we should require that the WADA labs do a reasonable job of method validation, but we should expect that “perfect” validation is probably beyond what we have a right to expect from any lab.

Up to the Introduction; back to part 5; on to part 7