Larry's Curb Your Anticipation, Part 10:
Specificity/Selectivity meaning in the ISL
Up to the Introduction; back to part 9; on to part 11.
We left our discussion in part 9 on the varying language in the ISL about specificity/selectivity, wondering how you can tell what really applies. We take note of an interesting distinction proposed by Mr. Idiot, but aren't sure it would be accepted.
[MORE]
I think that the ISL rules governing specificity are best understood in the context of our discussion of method validation. In this context, it is clear that selectivity must play a role in method validation, notwithstanding any permissive language we might find in ISL rules 5.4.4.2.1 and 5.4.4.2.2. It would make no sense to devote great effort to the determination of the accuracy of a test method, without devoting a roughly equivalent effort to determine that the method is measuring what it is supposed to measure. Understanding what we do about “fitness for purpose”, it would be impossible to argue that a test method lacking in selectivity could possibly be “fit for purpose.”
At the same time, it may be unreasonable to expect that WADA lab method validation should ensure complete selectivity. As we’ve stated repeatedly, the ISL is based on the assumption that standard methods are not available for doping testing, and that WADA labs will be required to develop their own test methods in house. It’s probably unrealistic to expect new and novel test methods to achieve perfect selectivity (notwithstanding the importance to the athletes being tested that the labs avoid making any mistakes). In addition, given that a urine sample is a complex matrix that might contain any number of unexpected substances, it might be impossible to prove that a lab method was capable of avoiding all possible interferences. Finally, we should recall that method validation is a finite effort, designed to determine fitness for purpose, but limited in important respects by factors of time and cost. Given all this, it seems most reasonable to define “specificity” for purposes of the ISL as a high degree of selectivity, but probably not as 100% selectivity.
But what of the difference highlighted above between the standards for avoiding (or limiting) matrix interference in ISL rules 5.4.4.2.1 and 5.4.4.2.2? Here on TBV, Mr. Idiot has advanced a good explanation for the difference in these standards. ISL 5.4.4.2.2 addresses the requirement to limit interference in testing for threshold substances, where the lab is required to measure the amount of the substance present in the standard. In contrast, ISL 5.4.4.2.1 governs the requirement to avoid interference in testing for non-threshold substances, where the lab is only required to determine the presence of the substance. Arguably, the standards for methods that need to measure the amount of a substance must be more exacting than the standards for methods that need only to detect the presence of a substance.
But I would caution against making too much of the difference in language between the standards in ISL rule 5.4.4.2.1 and 5.4.4.2.2. Recall what we’ve said so far about method validation. The goal of method validation is to ensure that a lab method is fit for purpose; a lab method that does not achieve good specificity (whether specificity is determined by avoiding interference or by limiting interference) is not fit for purpose. Moreover, given the unpredictability inherent in matrix interference (at least when dealing with a complex matrix such as urine), we can also expect that a lab will use the best means available to combat interferences, regardless of whether the lab is testing for a threshold or a non-threshold substance. At the same time, we expect labs to perform only a limited amount of method validation, so that method validation can be completed within a reasonable time and at a reasonable cost. In other words, we can predict that the main limitation on determining selectivity under the ISL will not be the standards set forth under ISL 5.4.4.2.1 and 5.4.4.2.2, but instead will be the limited effort required of labs under the ISL and ISO 17025 in performing method validation.
So, I think that the best way to understand the ISL rules on specificity is the way we’ve described above: under the ISL, specificity is a high degree of selectivity, but probably not perfect selectivity.
Up to the Introduction; back to part 9; on to part 11.
5 comments:
I'm still with ya' Larry. You nicely clarify the meaning of specificity.
I know you are avoiding the details of the case, and maybe you are going to say more about specificity, but I don't think your treatment fully accounts for the way 5.4.4.2.1 was actually used by the arbs, Suh, and even Ayotte. The arbs certainly acted like 5.4.4.2.1 applied to the actual testing, and that a violation of it (were it not for the permissive language) could constitute an ISL violation. Throughout the hearing, Suh placed a HUGE emphasis on matrix interference as applied to the actual tests, and the arbs seemed to accept the criteria, although they let LNDD off the hook because of the permissive language.
I continue to believe that (at least in practice, if not legal framework) specificity (at whatever degree of selectivity) applies to the actual tests.
syi
syi raises an interesting point that I've been wondering about.
USADA seemed happy to put together a case which was not constrained by relevance to the applicable WADA rules. In fact much of their "evidence" in the form of certain witnesses appeared to have no relevance to the case, let alone specific rules.
Should the Landis camp be constraining themselves to evidence which can be traced directly to a particular rule violation, or should they just put together the most convincing arguements they can, regardless of whether they strictly adhere to a rule violation.
I guess what I'm trying to say is which is more important, being able to say a particular rule has been broken or trying to establish that there is a high probability that the results are incorrect.
syi, I'm glad you're still with me. If there's a single piece of the opus that I'm most happy about, it's how I've finally resolved the meaning of specificity within the 5.4.4.2 rules. At least in my own mind, I feel like I finally understand what the rule requires.
The question you raised about what the arbs did the 5.4.4.2.1 IS addressed later on in the article (maybe part 12, though the discussion begins in part 11). Actually, it's the last thing I wrote before TBV wrenched the project away from me. It was probably the hardest part of the opus for me to write, and I could not write it until everything else was written and in place. Because you're quite right, the way I describe the rules governing specificity does not jibe with the way the rules were handled by the arbs. So, let's hold our discussion on what the arbs did with 5.4.4.2.1 until the entire opus is published.
The point you've raised about Suh's strategy is probably beyond the scope of the opus. That's a more complicated question that goes beyond the meaning of the WADA rules, and addresses the overall strategy behind the FL defense. Remember that a defense strategy does not have to be based on a single monolithic theory of the facts and the law. The defense can argue multiple theories, alternate theories, even contradictory theories. The ultimate goal is to win, and winning may require attacking by land, sea and air. If you can shoot enough holes in the argument made by the prosecution, then who really cares whether the holes actually add up to a single point of view? Please understand, I give Suh a lot of credit, I think his defense of FL was brilliant stuff. In any event, I'd ask that we hold off discussing what Suh did until the opus is finished, AND we've discussed what the arbs did.
Ali, that's a great question. That's exactly the kind of question I was hoping people would ask themselves as they read the opus. In a sense, your question is even more difficult than the one asked by syi.
Maybe the most difficult part of your question is when you ask what is more "important". I hate to do the lawyer thing, but your question turns on what is "important" to you. My guess is that what's most important to you is the truth, in which case the answer is easy, the results are more important than the rules. If what's most important to you is to win, then the answer is more difficult to come by, because the answer turns on what is going to persuade two arbitrators to side with you.
When you ask whether the FL team should simply focus on the rules or else make the most convincing arguments they can ... Well, they can do both, and I think they did do both. The more difficult question is, what's going to be persuasive to the arbitrators? It's a difficult choice. The arguments that focus solely on the rules appear technical to us -- they don't seem to address the question of whether FL doped. And you'd be right to wonder whether any purely technical arguments have a chance of persuading 2 arbitrators to exonerate FL -- you can imagine the criticism the arbitrators would have faced if they ruled (in effect) that FL probably doped but that he should escape sanctions because Claire Frelat used white out on her lab reports. But the arguments that go to the substance -- arguments about the quality of the chromatography, for example -- suffer for the lack of bright-line rules that tell us what is and what is not good enough. In effect, it seems like the issues we want to address (the more complicated science kinds of questions) are too complex to be addressed by bright-line rules, and the rules that we'd want to apply (the ones that tell us what a lab can and cannot do, in clear language without any wiggle room) address issues that don't go to the heart of the case.
One of the things I tried to do in the opus is to tie the rules and the science together, so that the rules DO address the issues we care most about. Let's hold until later a discussion of whether and to what extent I succeeded in doing what I set out to do.
(By the way, I DO want to get back to you about the m45 offset question, just haven't had the time.)
Larry,
Thanks for that response. You've given me a comfort factor on this series which I didn't previously have (I wasn't sure where it was going)
Cheers, Ali
Ali, when the series ends abruptly a part or two from now, I'll be expecting the "comfort factor" you mention to more or less evaporate.
But if that doesn't happen, perhaps you'll find a way to communicate the "comfort factor", so I can share it too!
Post a Comment