Larry's Curb Your Anticipation, Part 13: Misapplication

Up to the Introduction; back to part 12; on to part 14.

By Commenter Larry

At the same time, there are some things that would be misapplication of method validation rules to the assessment of results. The Majority of the AAA Panel seems to have made mistakes of this sort.

[MORE]

• Can the criteria for method validation be taken outside of the context of method validation, and used to judge the validity of a test result? The answer to this question seems to be a qualified “yes”. For example, in the FL case, FL argued that the LNDD violated ISL Rule 5.4.4.2.1 by failing to properly generate chromatograms that avoided matrix interference. During the arbitration hearing, both sides called expert witnesses to testify to the quality and reliability of these chromatograms. Predictably, the witnesses for USADA stated that these chromatograms were reliable, and the witnesses for FL disagreed. The majority of arbitrators ruled that these chromatograms “perhaps could have been better”, but ultimately found that the chromatograms were “fit for the purpose” and that the chromatography “unquestionably indicates the presence of exogenous testosterone in [FL’s] “A” and “B” samples.” In reaching this ruling, the majority arbitrators made the following statement, a statement that (in my opinion) completely misconstrues the nature, purpose and intent of the ISL:

In applying the language of the ISL what is required is that the “method should avoid interference.” The language is not mandatory. Had the drafters intended that matrix interference be avoided it would require wording such as “shall” or “must”. For this Panel to accept the submissions of the Respondent that matrix interference must be avoided would be a misconstruction of ISL 5.4.4.2.1. Dr. Ayotte confirms this statement in noting that a laboratory does not violate Article 5.4.4.2.1 of the ISL just because it produces a chromatogram that contains matrix interference. Therefore, even where matrix interference has occurred in the Stage 17 chromatograms it would not amount to a violation of the ISL. (FL decision paragraph 240.)

The majority’s reasoning here illustrates the danger inherent in reading rules out of context. The majority managed to interpret ISL Rule 5.4.4.2.1 to exclude any requirement to consider matrix interference – a reading that is possible only by failing to read the entire rule! (As we’ve seen, this rule includes a requirement that lab methods must achieve specificity, and the concept of “specificity” includes within it the need to avoid matrix interference.) But the majority’s reasoning is faulty on a more fundamental level, as the majority never considered ISL Rule 5.4.4.2.1 as applying to the lab’s method validation effort. In the context of method validation, a statement that a method “should avoid” matrix interference takes on a different meaning than the one found by the majority arbitrators. In the context of method validation, this statement would be read in light of the overall goal of method validation: to make a reasonable effort (within the constraints imposed by the cost and time required for the effort) to determine that a method is “fit for purpose”. In this context, the “should avoid interference” language in ISL Rule 5.4.4.2.1 might be interpreted to mean “do the best you can to avoid interference” or “make a reasonable effort to avoid interference”, consistent with the overall principle that a method cannot possibly be “fit for purpose” if it cannot provide us with a reasonable assurance that the method is measuring what we want it to measure.

So, let’s return to the question posed at the beginning of this bullet point: can the criteria for method validation be taken outside of the context of method validation, and used to judge the validity of a test result? The answer is “yes”, at least with respect to some of these criteria, but we need to be careful in how we do this. To be certain, if we want to assess the validity of a particular test result, we would want to consider criteria such as accuracy, uncertainty, specificity and interference. But we should avoid applying the specific language of ISL method validation rules embodying these criteria to our assessment of a particular test result. Instead, we should look to other rules within the ISL and ISO 17025 – rules that were written for the purpose of assessing a method result rather than the method itself – to provide us with the necessary guidelines for determining the validity of a particular method result.

Up to the Introduction; back to part 12; on to part 14.