Saturday, October 27, 2007

Larry: ISL Violation any way you look

Tuesday, Commenter "M" contributed a post arguing the Majority was correct. In the many comments to M's post, which are worth reading, Larry has added this epic, and it seems time to switch to a new conversation. -TBV


M -

So you'd like to do the legal analysis involving IRMS identification? Good! This is really very simple ...

The simple explanation

  1. The ISL includes TD2003IDCR. See ISL paragraph 1.0.
  2. TD2003IDCR requires each WADA lab to "establish criteria for identification of a compound." Once established, the criteria becomes part of the ISL for the lab.
  3. I can find nothing in the transcripts, briefs or other publicly available material to indicate that LNDD ever established such criteria. The failure to establish such criteria is a departure from the ISL. Alternatively, the criteria established by LNDD for identification of compounds in IRMS testing failed to satisfy TD2003IDCR, and that is also a departure from the ISL.
  4. Under WADA rule 3.2.1, USADA was required to prove that this departure from the ISL did not cause the adverse finding.
  5. USADA failed to meet the burden of proof referred to in (4) above.
Case dismissed.

The longer explanation

My guess, M, is that you're not going to be satisfied with my simple explanation. You're going to want to see something more complicated. TBH (to be honest), I think that the simple explanation works just fine, and that any more complicated explanation requires me to make a better case for USADA than they made themselves. But I'll give it my best shot.

[MORE]


(1), (2) and (4) above are just restatements of the rules, which we've already discussed. (3) and (5) are relatively obvious findings of fact that emerge from a reading of the trial transcript. It will take two posts to fully describe these facts. We'll start with point (3) above, to show the ISL departure based on LNDD's failure to establish and follow a satisfactory set of criteria for identifying compounds. I'll turn to the burden of proof issue under (5) in a later post.

LNDD had no criteria for identifying IRMS compounds

Did LNDD have established criteria for identification of compounds as required by TD2003IDCR? I only wish that someone had asked that question directly during the trial. But there's a great deal of indirect evidence that LNDD had no such established criteria. If such criteria existed, they should have been included in LNDD's standard operating procedure (SOP). The SOP does not appear to be available to us (it would be in French anyway), but the two LNDD lab technicians (Frelat and Mongongu) both testified in detail regarding the steps they took under the SOP when they performed the IRMS analysis. How did they identify the peaks shown on the IRMS graphs for the FL samples? I can find not one word of testimony from the lab technicians on this point, not in hundreds of pages of transcript.

Let's look at the testimony of Cynthia Mongongu, beginning at page 390 of the transcript (271 of the pdf file). Ms. Mongongu was the lab technician who performed the IRMS analysis on the Stage 17 A sample (Esther Cerpolini did the GC/MS), and verified the analysis of the B sample. Her testimony was longer and more detailed than the testimony of Ms. Frelat, plus Ms. Mongongu was the more experienced and knowledgeable lab technician, so I've focused more on her testimony. This testimony is lengthy (about 105 pages) and focused almost exclusively on how she performed the IRMS analysis. From her testimony, she took the following steps:

a. She prepared the sample, following the LNDD SOP.
b. She performed the GC/MS analysis for the identification of the compounds. According to her testimony, she verified that the GC/MS instrument is operating properly, by first making certain that the MassLynx is functioning properly and the ion source is functioning properly, then she checked the instrument for leaks, and she injected a mixed acetate into the GC/MS to make certain it was operating properly. Once the instrument was verified, she injected the prepared FL A sample, in fractions, in a prescribed sequence along with fractions of blank urine samples.
c. She performed an IRMS analysis. This involves verifying that the instrument is set up to verify the CO2, and that the instrument is stable. Then the instrument is calibrated with reference to a test on the mixed cal acetate. Finally, FL's sample is injected into the IRMS.
d. She performed a data reduction process on the data, consisting of verifying the background noise, and verifying that the chromatograph peaks are correctly integrated. This is the point at which Ms. Mongongu would make manual adjustments to the data.
e. At this point the results of the test are calculated, and it was determined that FL's A sample was positive for exogenous testosterone.

Please tell me, where does Ms. Mongongu describe how she identified the IRMS peaks? She does not do so.

On cross-examination, Maurice Suh walked through all of these steps again with Ms. Mongongu. (see testimony beginning on page 556 of the transcript, page 421 of the pdf file) If, somehow, the work Ms. Mongongu had performed to identify the IRMS peaks was omitted in direct examination, Ms. Mongongu had the opportunity to provide this information during cross-examination. In re-examining the steps taken by Ms. Mongongu, Mr. Suh was careful to walk through these steps in sequence, making certain that no step was skipped. So, Mr. Suh started with the stability runs on the IRMS machine, and then asked Ms. Mongongu to describe each step she'd performed, in sequence, using questions structured like "After you did x, then you did y, correct?" Again, there's not a word regarding how Ms. Mongongu identified the peaks in the IRMS tests.

You can also examine the testimony of Ms. Frelat, which is shorter but substantially the same as the testimony of Ms. Mongongu.

It's worth noting that Dr. Meier-Augenstein also wondered out loud how LNDD went about identifying its IRMS peaks. On direct examination (transcript pages 1418-19), Suh asked Dr. M-A: "When you look at the document package, do you see any documents which help you identify how they decided which one of these peaks constitutes the internal standard?" Dr M-A responded "No."

The cross-examination of Dr. M-A is even more damning to USADA's position, which is surprising given that USADA appeared to INTENTIONALLY use this cross-examination to prove that LNDD had no criteria for identifying compounds. When Dr. M-A tried to argue on cross-examination that LNDD must have had SOME procedure for the identification of the IRMS internal standard peak, USADA's counsel seemed intent on proving the contrary (testimony p. 1517, pdf 1291):
"Q. And where is it in the Paris laboratory's specs that says when you use the internal standard in connection with identifying retention time in a blank urine or athlete sample that it has to be within a particular delta value spec?"


Dr. M-A eventually responded as follows (transcript p. 1518, pdf page 1292):
"A. I -- I don't know. I don't know how they do it. Because, as I would say, there's nothing in the specs, so I don't know. Is it divine intervention or they just pick one? How do they -- how do they know which peak is their internal standard? They must have -- they must have some -- some criteria to say, Oh, let's see, there's one peak at this time; there's one peak at this time; that's not the peak at this time. They're also 20 seconds apart. How do they choose which of those it is? I don't know. And you're quite correct. There's nothing in the specs that actually tell you how to do it."


At this point, USADA's counsel could have easily rebutted Dr. M-A and shown him that LNDD DID indeed have criteria for identifying the internal standard. USADA did not do so. They appeared intent on making the point that no such criteria existed at LNDD.

I don't see any other testimony relating to whether LNDD has the established criteria required by TD2003IDCR. The testimony of other witnesses (like Dr. Brenna) did not speak directly to what LNDD did or what procedures LNDD had in place to identify compounds. Instead, this testimony spoke to what are acceptable identifying techniques as a general matter. (This other testimony will become relevant in a later post, when we look at whether USADA effectively met its burden of proof under Rule 3.2.1.) You can read this other testimony, and perhaps you can point out to me whether or where these other witnesses spoke to what LNDD actually did. In any event ... the testimony of Ms. Mongongu and Ms. Frelat is the best evidence of LNDD's actual practices and policies, though the testimony of Dr. M-A cited above IS instructive.

I think my conclusion here is a fair one: if Ms. Mongongu or Ms. Frelat took ANY steps to identify the IRMS peaks, these steps were casual and not systematic. There's no evidence that Ms. Mongongu and Ms. Frelat followed anything like "identification criteria" to identify these peaks. I conclude from this that in all likelihood, LNDD had no such identification criteria. LNDD's failure to adopt identifying criteria under TD2003IDCR is a clear and obvious departure from the ISL.

If LNDD had identifying criteria, the criteria failed to meet TD2003IDCR standards

But ... let's assume I'm wrong. Let's assume that LNDD DID establish TD2003IDCR identification criteria. What might that criteria be? Before we get started on this part of the analysis, I'll issue a warning: even if we assume that the LNDD had established TD2003IDCR criteria, the evidence of what this criteria might be is skeletal and not entirely consistent. I STILL think the best evidence is that no such criteria existed. However, if there WAS such criteria ... the best evidence I can muster is that any such criteria required a relative retention time analysis.

In her testimony, Ms. Mongongu testified on two occasions that LNDD computed included an internal standard in its testing in order to calculate relative retention times. (testimony p. 434, pdf p. 311; testimony p. 653, pdf p. 509). Here is one quote from her testimony:
" Q. Why do you add an internal standard to the blank urine and the athlete's sample?
A. The internal standard allows us to calculate the relative retention time of the molecules analyzed.
Q. Is that to make sure that you're looking at the right peaks?
A. Absolutely, yes."

You can also look at the IRMS SOP described above. The second step under that SOP is to perform a GC/MS analysis. What would be the point of doing such an analysis? The science types can weigh in here, but in the context of an IRMS test, a GC/MS analysis would be performed to provide peaks that could be used to identify the compounds being measured in the IRMS test. I don't know why else a GC/MS test would be relevant, other than to provide retention times that could be compared in some manner to the retention times for the IRMS test.

Dr. Ayotte's testimony provides corroborating evidence. On direct examination, Dr. Ayotte testified that she understood from Ms. Mongongu's testimony that LNDD ran a 5aA internal standard test, and that this was a good practice "to determine the relative retention time of your analytes." She went on to testify that this testing is "a necessity, otherwise, you don't know what you are measuring." (testimony pages 811-12, pdf pages 652-53) Dr. Ayotte testified more than once regarding the link between the internal standard test and the determination of relative retention times (see, for example, testimony page 849, pdf page 686).

So, if LNDD DID have established TD2003IDCR identification criteria, this criteria was based on relative retention times. The next question is, what procedures were established under these criteria for comparison of relative retention times? On this question, we're stymied. As I've already stated, there's not a word of testimony on what LNDD did to identify the IRMS peaks. However, the assumption (in the majority opinion and on this forum) is that LNDD identified the IRMS peaks by comparing them visually to the peaks identified in the GC/MS test. The best explanation I can find for this technique was provided by Dr. Brenna in his cross-examination (transcript page 1971, pdf page 1702):
Brenna: You can't use relative retention times. I explained why this morning. Would you like to know what to use? I can explain it. We have the chromatograms up there.
Suh: No, that's all right.
Brenna: Okay.
Suh: Actually, no, why don't you go ahead and explain it. Go ahead.
Brenna: Well, on the GC/MS side, we see a pattern, so we can see peak heights. And so -- and we want to look at the overall pattern is what -- an intermediate-sized peak; a small peak; this is one of the strong peaks; and then a large one. And then we move over a bit, and we find a large peak, an intermediate peak, a smaller peak. And then we move to the end, and we see a large peak. And if you look, you see the same pattern here. In fact, I would -- I would say, you see a number of peaks here, a number of peaks here, a number of peaks here, and they have approximately the same -- same look from this distance. And so -- and so I probably stuck with that peak, and say, well, that peak looks -- of course, I'm already anchored on the internal standard, because I do know what that retention time is. And I then can identify that last peak, again, based on pattern, and these center peaks, based on pattern. And that's one of the ways that I would -- one of the ways that I would identify peaks, comparing them.


We have huge problems relying on this description. First, Dr. Brenna is NOT expressly describing what went on at LNDD. He's saying how HE would identify the peaks (presumably, based on the limited data he had in front of him -- he never said that this is the method he uses in his own lab). Plus, he's saying expressly that this identification method is NOT a method for measuring relative retention times. This is a problem, given that if LNDD had established IRMS "identification criteria", our best evidence is that this criteria WAS a relative retention time analysis. (Personally, I understand Dr. Brenna to say that the method he's describing is not the kind of relative retention time analysis performed by Dr. M-A; instead, it's a more relaxed and fuzzy kind of RRT analysis.) So, it's difficult to say that Dr. Brenna is describing the LNDD's identification criteria. (Again, I see this as proof that LNDD HAD no such identification criteria -- if they had such criteria, we should at least be able to figure out what the criteria was.) However, I have nothing else to go on. So ... in the absence of anything better, let's say that if LNDD had "identification criteria" as required under TD2003IDCR, this criteria was a relative retention time analysis, performed in the manner described by Dr. Brenna.

The next question I'll address here is: assuming the LNDD identification criteria are those described by Dr. Brenna, do they meet the requirements of TD2003IDCR. I think it's pretty obvious that they do not.

We can start with a definition of "criteria". My old Oxford-American dictionary defines "criterion" as "a standard for judgment." This is also the Wiktionary definition, more or less. So, a "criterion" is a kind of "standard". I'm not a scientist, but I think that the very essence of a scientific criterion or standard is that it allows for objective review and evaluation. A "criterion" has to be sufficiently descriptive and detailed to permit person A to determine the validity and accuracy of a judgment reached by person B. If a procedure to reach a judgment is subjective, so that two people can reasonably reach different conclusions while following the procedure, then the procedure is not a "criterion".

My definition of "criterion" is supported by a review of the procedures listed as acceptable criteria under TD2003IDCR. All of these criteria are detailed, objective and independently verifiable. There's little or no "wiggle room" under any of these criteria.

Now M, you may argue that it's difficult to create objective and scientific criteria for the visual comparison of two different graphs. It may be difficult, but it's not impossible. As an example, please take a look at WADA's own TD2007EPO, which contain objective criteria for the visual comparison of EPO results: you divide the result chart into three parts, you determine which two bands are most "intense" based on a densitometry measure, and so forth. It's an objective standard. You can use this standard to review the judgment reached by a lab that an athlete doped with EPO.

TD2007EPO is a good reference point for us to use when we look at the methodology for identifying compounds as explained by Dr. Brenna. Under the Brenna method, you identify a pattern of "large", "intermediate" and "small" peaks on each chart, and then you check to see if these two patterns are "approximately the same" from "this distance." Does the Brenna methodology come even remotely close to describing a "standard" that person A could use to check the work performed by person B? How do you distinguish a "large" peak from an "intermediate" peak? How do you determine whether two patterns are "approximately" the same? And what in the world did Dr. Brenna mean when he said that you'd compare the two patterns "from this distance"? How can you possibly use the Brenna methodology to review the judgment reached by a lab technician? You can't.

M, if you look at the Brenna methodology for identifying compounds, the methodology does not even remotely look like a criterion or a standard. As I said, I think that the best explanation of Brenna' testimony is that LNDD did not HAVE a standard for identifying IRMS compounds as required by TD2003IDCR, and that Brenna did the best he could under the circumstances to justify what LNDD did or failed to do. The only alternative to this conclusion is that the LNDD's method of identifying compounds is the one described by Brenna, and that this method fails miserably to meet the ISL requirement that labs have "criteria" in place for such identification.

Of course, we could also examine whether the Brenna methodology, which is the only description we have of the steps LNDD might have performed to identify the FL IRMS compounds, is an adequate methodology to measure the relative retention times that LNDD said it used (according to Ms. Mongongu and Dr. Ayotte) to reach its doping finding. According to the only evidence we have on how to measure RRTs, which is the testimony of Dr. M-A, the Brenna methodology is invalid. And Dr. M-A’s testimony here is supported by Dr. Brenna (as well as the majority decision), who proclaimed that it’s impossible to measure RRTs across two different GC machines. I think this is the point that tbv and others have made here: the methodology described by Dr. Brenna and endorsed by the majority decision is unsound as a matter of science. But from a legal standpoint, we can make essentially the same point, but in a simpler manner that I think is impossible to refute: whatever you might think of the scientific validity of the methodology described by Dr. Brenna (and OMJ), the methodology is not a “criterion”, and the methodology thus fails to meet the ISL requirements.


[BACK FROM MAIN BODY]

Under either alternative, we've proven a departure on the part of LNDD from the ISL, and we’ve proven that the majority decision was incorrect on this point. The next step will be to examine whether USADA effectively rebutted the presumption that this ISL departure caused the doping finding in the FL case, which is the only way left open at this point to defend FL’s doping conviction.

I'll pause at this moment for comments and to catch my breath.

26 comments:

Unknown said...

Holy Cow.

Now this is how the arbs decision (both for and against Floyd) should have read.

Sure opens a whole new can of worms. I hope Floyd's legal team is reading this for their appeal.

Mike

Larry said...

Mike, thank you. That's very high praise. FL's legal team is one of the best I've ever seen. Suh in particular.

Credit where credit is due: M got this discussion started at a very high level. I'm just trying to follow M's lead.

tbv, thanks for giving my comments such a high profile. Not to mention the editing and the nice formatting.

Larry

Unknown said...

Your Welcome Larry!

We all said from the beginning that the arbs decisions would have to be very clear to justify their decisions. I felt like when I was reading the Majority Decision, they were trying to confuse the decision. I couldn't follow it and it didn't make much sense to me. It seemed more like a scientific paper, yet wasn't in depth enough to understand. The decisions needed to be written so a first grader could understand them.

Campbell's dissent was easy to understand, but I felt it lacked 'depth'.

Mike

m said...

Larry,

Good work.

This is a cursory rebuttle to your arguments, since I haven't had time to research the record.

You argue that LNND violated TD2003IDCR because it did not 1) utilize or, 2) establish, sufficiently well defined GC identification criteria.

1. As, to your first argument, Landis has the burden to show an IS violation, that is he has the burden to show that there was no identification criteria. Your first argument seems to assume that the burden is on USADA to show exactly when and where the identication was made. This is incorrect. The fact that we can't find testimony by Mongongu et.al. describing the identification process in specific detail is not sufficiently conclusive to show that no criteria were used, since that issue might not have been thought important by the parties and not included in the questioning. Clearly from the exhibits an identification of peaks was made and the carbon ratios calculated. The legal presumption is that the proper criteria were used to make that identification. And as you have noted, both Mongongu, Ayotte, and Brenna state that RRTs were used to identify the peaks. Although, Brenna makes clear however, that the RRTs don't have to meet the 1% criteria. So the first prong of your argument must fail, because Landis can't carry his burden here.

(You say that you can't find the SOP. I recall that some 1600 pages of document were provided that never were entered into evidence, and perhaps not all evidence is available online. So I wouldn't be surprised if it could be out there.)

2. Your second argument is that, assuming that LNND had identification criteria, those criteria were not sufficiently detailed or objective for scientific replication. You conclude from the testimony that the method of identification was the use of RRTs and visual matching of peaks. I think your argument here substitutes a lawyer's idea of what is sufficiently specific and replicable, when the true criteria should be the scientific testimony as to that issue. What may appear insufficient to you may be perfectly fine for other scientists. Again the presumption is that whatever standard was used was adequate. Landis and you have the burden to show that the standard used was not specific enough. This was not an issue in the case, and there was little or no testimony with respect to it, only with respect to whether the identification violated the 1% standard.. If I were a judge, I would want scientific testimony on this and would not want to substitute my own judgment. The arbs were comfortable enough with ID procedure described by Brenna and Ayotte. Since you are not even sure what criteria was used, and have to construct a criteria, I don’t think you can carry your burden of showing that the criteria was not specific enough.

Nevertheless, let me address your argument. You claim that it appears from the testimony of Ayotte, Mongongu, and Brenna that LNND used the sequence of relative retention times and comparison of the sequence of peaks from a known internal standard, 5A Androstanol, to identify the metabolites. Brenna testified that the 1% standard did not apply.

TD2003IDCR says: Examples of acceptable criteria are..."For capillary gas chromatography, the retention time (RT) of the analyte shall not differ by more than one percent or +/- 0.2 minutes (whichever is smaller) from that of the same substance in a spiked urine sample, Reference Collection sample, or Reference Material analyzed contemporaneously. In those cases where shifts in retention can be explained, for example by sample overload, the retention time criteria may be relaxed.”

Assuming that this criteria also applies to relative retention times, TD2003IDCR specifically states that the retention time criteria may be relaxed. So retention times may be used, but don’t have to conform to the 1% standard. This is what occurred in this case. For example, in the F3 sample, LNND used the relative retention times to identify which peak in a sequence of 4 central peaks were the 5b and 5A metabolites. An internal standard was accurately measured (there is no dispute about this), and the relative retention times from that internal standard told us that the 5B came after the first small peak of the 4 central peaks, and that the 5A metabolite came after the 5B. The retention times allowed us to determine the sequence of peaks and to compare those peaks. Some people claim that you are not allowed to look at the chromatographs and peaks in making your identifications. This doesn’t seem right. Retention times correspond to peaks. All the witnesses claimed that good chromatography was central to the identification and measurement process, and much time was spent on the height, shape and location of peaks. Using the RRTs and looking at the chromatographs it’s clear to OMJ and others that the 5A and 5B were properly identified.

So if we take the language of TD2003IDCR as our identification criteria and standard, then it was met in this case.

One could argue that if the 1% standard can be relaxed, the lab must specify how much it can be relaxed. I would argue that it is up to Landis to prove this if anyone is, not the lab.

My perspective is different from Larry’s. I believe that the identification was scientifically accurate, so I am not looking for technical arguments to invalidate the identification standards.

3. One has to wonder why if this is such a simple and slam dunk argument then why didn’t Landis' very expert lawyers raise it in the arbitration. But we shall see. -)

Laura Challoner, DVM said...

m,

You say,

"3. One has to wonder why if this is such a simple and slam dunk argument then why didn’t Landis' very expert lawyers raise it in the arbitration. But we shall see. -)"

The answer is that the pivital point upon which the case ultimately turned was unknown to the very expert lawyers on the Landis team, who would have had to have been flys on the wall when Botre wrote the opinion and then transformed themselves back in history to address the issue specifically at hearing with the volumn you seem to rquire, as the monkies danced to Matt Barnett's tune.

As someone who actually has written final decisions, I simply state that were I to decide a case upon a point inadequately addressed by the parties, I would invite both parties to address the issue specifically, especially if I held open the hearing as was done here.

The opinion was written backward; The result desired required certain findings, foremost among them was a declaration that no ISL violations occured. That required a rational for every single allegation. The buden turn had to be avoided, a lesson learned from the Landaluze case.

m said...

Bill,

"I simply state that were I to decide a case upon a point inadequately addressed by the parties, I would invite both parties to address the issue specifically, especially if I held open the hearing as was done here."

Bill that assumes that the arbs, including Campbell who was looking for arguments to support Landis, ever thought this was an issue. Clearly they didn't, because even Landis's side didn't raise it. The Landis attys took their shot at the identification issue with the 1% standard and the poor chromatography arguments if my memory serves me. They could have thrown other arguments up against the wall, but didn't. Sure they might have missed something. But the fact that they missed this issue tells me maybe they didn't think it was valid.

Laura Challoner, DVM said...

m,

Campbell wasn't in an intellectual position to argue with Botre and I wouldn't have been either. We are more than a month past the decision and some elements are finally crystalizing among a bunch of really smart people.

Campbell had 10 days and less than that, actually, to try to bring such matters into focus. Instead, consistant with his role, he took on the system's total lack of fairness. I don't blame him for that, as the system deserves the condemnation it got from him.

The process as well as the result was disturbing. For me it was like watching Mr. Potter work in "It's A Wonderful Life", especially when he stole Uncle Billy's bank deposit and then picked up the phone to swear out a warrant on George Bailey because the deposit wasn't made.

The majotity did not seak justice nor truth. It resolved to do what those Panels are destined by design and mathmatics of their 2 to 1 majority to do; "convict" the athlete. You give them way too much intellectual credit.

Larry said...

M -

Great post. Seriously. I only hope that when FL's team argues this case on appeal, they don't have to face arguments as strong as the ones you've raised.

You are a skilled advocate making the best case possible for USADA under these circumstances. Luckily for me, the circumstances don't allow you to make much of a case! (IMHO)

I'm going to try to keep this brief (at least by my standards), so that the salient points do not get lost under a mountain of factual information.

1. Yes, the burden of proof is on the athlete to prove a departure from the ISL. But the burden is not as high as you make it out to be -- the athlete has to prove a departure by the preponderance of the evidence. I've more than met this standard, by showing:

(a) in the course of detailed testimony by LNDD personnel, going through the steps taken in the IRMS analysis on a step-by-step basis (and taking into account the time each step took to perform), no mention was made of any procedures used to identify peaks;

(b) USADA's witnesses contradicted each other when they described how LNDD went about identifying IRMS peaks; and

(c) Dr M-A testified that LNDD had no method to identify IRMS peaks.

(d) There's no evidence to show that LNDD had TD2003IDCR criteria in place. So you have no evidence to counter my evidence.

I'll admit, this is no "smoking gun". But it's more than enough to prove, by a preponderance of the evidence, that LNDD had no TD2003IDCR criteria in place. Therefore, there's a departure from the ISL. I can go through the facts here again in more detail, if you like.

Let's look at your arguments. Yes, clearly LNDD identified the IRMS peaks, in that we see labels next to those peaks. But that's not evidence that they had "criteria" in place to label those peaks. Yes, there's a legal presumption that LNDD used criteria to label those peaks, but they lose that presumption once the athlete proves the presumption is incorrect by a preponderance of the evidence (as I think I've done). I did not mean to say that Mongongu, Ayotte and Brenna all stated that RRTs were used to identify the peaks; Mongongu and Ayotte so stated, but Brenna said it was impossible to use RRTs to identify the peaks under any possible standard (1%, 5%, 10%, whatever).

(No, I can't find the SOP, and I searched diligently for it. My wife is fluent in French and she owes me a favor! We'll have to proceed without the SOP)

2. As an alternative to my argument that LNDD had never adopted criteria per TD2003IDCR, I argued that the procedure used by LNDD to identify IRMS peaks did not rise to the level of "criteria" as required under TD2003IDCR. I'm having trouble following your argument that I'm wrong here.

You may be saying that I don't know what procedure LNDD used to identify these peaks. That's hard for me to argue with, as I don't think they HAD a procedure. However, let's look at the evidence. Mongongu and Ayotte said that LNDD used RRTs, but the overall evidence indicates that this must not be the case. Dr. M-A said that RRTs don't support the LNDD's findings, and Dr. Brenna said that you couldn't use RRTs under any circumstances. So, if the LNDD had "criteria", I don't think they were RRT criteria.

Brenna said that one could use pattern matching, and that appears to be the method endorsed by the majority decision. So for purposes of this analysis, I've assumed that LNDD used pattern matching, and I've asked the questiom whether "pattern matching" could rise to the level of "criteria". I don't think it can.

You want me to rely on the scientific testimony as to whether "pattern matching" is "criteria". However, I think the meaning of the term "criteria" is within the purview of us lawyers. Also, there was no scientific testimony on this point. Brenna never said that his "pattern matching" technique rose to the level of criteria. So I think it's up to the finders of fact to determine whether pattern matching is criteria.

On this question, all we really have to go on is the testimony of Brenna. He described the pattern matching technique as identifying big, medium and small peaks in two graphs and seeing whether they were approximately the same from a distance. How can that technique be a "criteria"? How can you expect that any two scientists utilizing this technique would reach the same results, or be able to justify their findings to a colleague? We've seen in the TD for EPO that you can write good criteria for the visual evaluation of a graph, but Brenna's not describing anything approaching the criteria in the TD for EPO. Brenna's describing a purely subjective look to see if one graph is approximately similar to another graph, and there's just not enough in what Brenna describes to provide a standard for how two different scientists should compare two graphs.

Sorry, M. I'll defer to the scientists, but only within the confines of the rules defined by WADA. "Pattern matching" is not "criteria". Even if Brenna had said that "pattern matching" is "criteria" (which he did not), that would not make it so.

M, in your post, you spend some time conjecturing that maybe LNDD had RRT criteria in place under TD2003IDCR, using a more relaxed standard than the 1% rule set forth in the TD for RTs. That's a pretty creative argument! However, we have no evidence of what that more relaxed standard might be, or even that there WAS such a relaxed standard in use at LNDD. And we have Brenna's testimony that there's no scientific basis for any such standard, no matter how relaxed.

Besides M, COME ON! We have a majority decision that turned on the acceptance of Dr. Brenna's testimony that you could not do an RRT analysis, that flatly rejected Dr. M-A's position that you CAN do an RRT analysis. NOW you're going to argue that there was no departure from the ISL because the LNDD had secret criteria in place for evaluating RRTs? That's a little too circular for my tastes.

You ask, if my argument here is so good, why didn't the FL team make the same argument? The answer is, I think the FL team thought it had better arguments. And I think they wanted to prove that FL didn't dope. I'm less ambitious. All I want to prove is that USADA could not properly convict FL under its own rules. Then, while FL continues his racing career in 2008, we can happily debate the science issues.

Now, you're still free to argue that regardless of the lack of TD2003IDCR criteria, the LNDD test results STILL prove that FL doped. But you do that in part 2 of our 3.2.1 discussion, when we get to the issue of causality, at which point the burden of proof will be on you.

Larry said...

M -

I'm going to see if I can restate my earlier post more succinctly.

We agree that LNDD was required to have criteria in place for the identification of IRMS peaks that satisfied TD2003IDCR ... and that they were required to follow these procedures. I'll go along for the moment with the idea that LNDD is presumed to have such criteria, and that FL had to prove by a preponderance of the evidence that it had no such criteria (or that it did not follow its own criteria) in order to prove a departure from the ISL under rule 3.2.1.

We then struggle with the issue of whether LNDD had any such criteria, and if it did, what the criteria might be. There are four possibilities:

1. LNDD had no such criteria.

2. LNDD had procedures in place requiring the identification of IRMS peaks by means of an RRT analysis.

3. LNDD had procedures in place requiring the identification of IRMS peaks by means of a pattern matching analysis, in the manner described by Dr. Brenna in his testimony.

4. LNDD had procedures in place for the identification of IRMS peaks, by means that were not disclosed in the arbitration.

I think that possibility number 1 is most likely. It's the only theory that explains the evidence: why Mongongu and Frelat never spoke a word about how they identified the peaks, not in hundreds of pages of testimony describing in detail all of the other things they did, when they did them and how long they took. It explains the confusion of USADA's own witnesses on how LNDD identified compounds (Mongongu and Ayotte testifying that LNDD used RRTs, Brenna testifying that RRT analysis could not be done at LNDD) -- there being no criteria in place for these witnesses to rely upon. It explains why Dr. M-A was unable to identify the criteria used by LNDD, why he stated that there appeared to be no such criteria in place, and why USDA never bothered to contradict his testimony.

Possibility number 2 seems highly unlikely to me. Admittedly, both Mongongu and Ayotte stated that LNDD used a RRT analysis. But USADA went through great pains to indicate that RRT analysis was impossible and unreliable across the two machines used by LNDD. So I doubt that LNDD could have adopted criteria utilizing RRTs. If I'm wrong and LNDD did have such criteria, then by USADA's own testimony, the criteria was unreliable and thus could not have possibly satisfied the requirements of TD2003IDCR. Or alternatively, LNDD had such criteria in place but failed to follow it in this case.

Possibility number 3 is more likely than possibility number 2, given the Brenna testimony. I thinkthat we are entitled to rely on the Brenna testimony as a complete description of what LNDD did to identify the IRMS peaks. But the pattern matching procedures described by Brenna do not rise to the level of "criteria", for all of the reasons we've discussed previously.

I think that possibility number 4 is an impossibility and can be dismissed out of hand. There was too much attention paid to this issue at trial to discover, at this late date, that LNDD has some secret formula for identifying IRMS compounds that never came out at trial. If LNDD used a secret formula for identifying compounds, then why did Mongongu and Frelat say that LNDD used RRTs, and what the heck was Brenna talking about? No, we have a right to rely on what the USADA witnesses testified to at trial. Moreover, we know pretty well what methods a lab COULD adopt for identifying compounds, and it does not seem that LNDD could have used any method other than RRTs or pattern matching.

So, on further review, we're down to possibilities 1, 2 and 3. Those are the only possibilities. I can't prove which of these possibilities represents the true state of facts, but I think I have proven by a preponderance of the evidence that one of these three possibilities is the true state of facts. And each of these possibilities is a departure from the ISL.

m said...

Larry,

General comment. You are trying to prove something like a negative, failure to use or adapt adequate identification criteria, on an issue that was not litigated in the case and on which there is only fragmentary evidence. This is very hard to do in any case. On top of that you are facing a legal presumption that LNND did in fact use and adopt such criteria.

Specific comments:

1. So as to the first prong of your argument, if I'm a fact finder I'm not going to throw out otherwise valid scientific results unless you can actually show me that the lab in fact used no identification criteria. I am not going to draw an inference from circumstantial evidence when the issue was not even litigated.

2. Regarding your 2) and 3), my response would be that from some of the testimony the lab appeared to use RRTs, "structural information", and visual inspection of the chromatographs to identify the metabolites.

Brenna's testimony while ambiguous can be reconciled. When he said RRTs couldn't be used on redirect and cross, he was referring to the 1% standard only, e.g. that one could not expect the RTs and RRTs to match within that standard. This is consistent with his earlier testimony that one would use RRTs and "structural information" to make the identification. Structural information could mean both sample preparation, i.e. you knew what metabolites were supposed to be in each fractionated sample F1, F2, and F3, and the sequence, location, height and shape of the peaks.

As I argued before, if we assume that TD2003IDCR was the identification criteria used, then I believe such an identification process meets that criteria, since the IS says the 1% requirement may be relaxed. As you noted in the thread about cortisone, the standards and criteria required by the ISL's can be very vague. I don't think you can substitute your legal notion of what is specific enough for scientific testimony on that issue. In the absence of such testimony, I think it unlikely that a fact finder would reach out and invalidate a technical standard as being not specific enough, especially when it is convinced from the scientific testimony that a valid identification has been made.

This is not to say that you might not be able to prevail on this issue in a new trial if there is actual scientific evidence in support of your view.

3. As to your 4) possibility, there is a statement in the majority opinion that indicates individually both the GCMS and the GC-IRMS complied with the 1% standard in TD2003IDCR. I haven't been able to confirm this, but if this is correct then one can argue that LNND complied with TD2003IDCR.

m said...

BTW,

I located some further retention time evidence that convinces me that the identification of the metabolites were accurate.

USADA 0185 shows that the IRMS retention times for the metabolites, 5B and 5A and Preg, in the Landis F3 sample were nearly identical to the IRMS retention times for the same substances in the blank urine. Blank urine contains natural testosterone and its metabolites, and the blank urine was fractionated in the same way as Landis' F3 sample, so should contain the same metabolites in the same sequence. The RTs match.

Unknown said...

M,

Does that mean that the IRMS found natural testosterone in Landis' sample or exo-testosterone?

Mike

m said...

Mike,

The IRMS found exogenous T in Landis' F3 sample based on the carbon ratios for the 5B metabolite. That same 5B metabolite (but from natural T) is also contained in the blank urine, and had the same retention time as the 5B metabolite identified in the F3.

It's possible that the blank urine could be considered a "spiked urine sample" within the meaning of TD2003IDCR, but OMJ said that this was not the case because testosterone occurs naturally in urine anyway, so there would be no need to spike it.

In any case, this shows matching retention times for the same metabolite, and provides verification that the 5B and 5A were accurately identified.

the Dragon said...

M,

Interesting...to use your dismissal of one of Larry's points, stating if it were so why didn't FL's attorneys make that point.

If your reading is correct, why didn't either USASA or the "Majority" take that point and run with it? Beat's their tortured reasoning, IMHO.

Regard,

David

Larry said...

M -

First and foremost, your work here continues to impress me.

You've raised a number of points that I'm going to need to address, in detail. I'm going to try and do this, point by point, with the detail necessary to forge common ground between us, or else to narrow and define the issues that separate us.

The first point I'm going to address is the Brenna testimony. I don't think you're characterizing it correctly. I don't think the Brenna testimony can fairly be read to support use of RRTs, with a more liberal standard than the 1% - 0.2 minute standard taken from TD2003IDCR and used by Dr. M-A.

It's going to take me some time to put this together, and right now my day job is intervening in a major way. I'd ask that you take a closer look at both rounds of Brenna's testimony, as well as the testimony of Dr. M-A (which provides the context for the second round of Brenna's testimony). In the meantime, I'll work up a more detailed summary of what Brenna said and what it means. As soon as I can find the time!

Mike Solberg said...

m, you were responding to a different Mike, but here goes anyway. You know this first part already, but just for clarity...

The IRMS did not find exogenous T in Landis' F3 sample based on the carbon ratios for the 5B metabolite.

The IRMS found SOMETHING (or SOME THINGS) eluting at "such and such" a time with a CIR of "thus and such" (-31.48 was it?).

Without the complete mass spec data, the IRMS itself cannot identify what substance(s) gave it a reading of thus and such.

Without the complete mass spec data, you have to relate the IRMS to the GCMS somehow (how to effectively do that is a point of much contention here obviously).

I don't argue, and I don't think many do, that there were no metabolites of testosterone in those peaks on the IRMS chromatogram of the F3. The question is, what else is there?

LNDD has provided no evidence that the peak which lead to a CIR of thus and such contained only metabolites of testosterone.

I believe they are required to do so.

ISL 5.4.4.2.1 says:

"Confirmation methods for Non-threshold Substances must be
validated. Examples of factors relevant to determining if the
method is fit for the purpose are:
(bullet) Specificity. The ability of the assay to detect only the
substance of interest must be determined and documented. The assay must be able to discriminate
between compounds of closely related structures."

Larry has raised the point that that part of the ISL is one step removed from actual lab process, and that is right. It is talking about the design of the process/assay. But look at this section of C. Ayotte's testimony about ISL 5.4.2.2.1:

"[Suh]Q. Okay. But setting aside the validation process for a moment, and talking about the actual analysis, the -- in the analysis, the lab -- just going through these bullet points, the lab still has to show that they properly identified the substance, correct?

A. Correct. That is why there's the
purposes of having controls in the assays as well.

Q. And during the analysis, the
laboratory still has to demonstrate that there was no carryover, correct?

A. Yes.

Q. And during the analysis, the
laboratory still has to demonstrate that there was no matrix interference, correct?

A. Correct.

Q. And if there is matrix interference in connection with the analysis of the sample,
that affects the accuracy of the results, correct?

A. Of course.

Q. And of course, if there's carryover in the analysis of the sample, that affects the
accuracy as well, correct?

A. Yes.

Q. So these points here don't just go to validation; these are also things that you need to look at in the analysis.

A. Yes."

So according to Ayotte, 5.4.4.2.1 does set standards for the actual conduct of the analysis, not just the validation of the process.

LNDD, because it has not provided the complete mass spec data, has not met the standard of "Specificity" under ISL 5.4.2.2.1. Which, again, says: "Specificity. The ability of the assay to detect only the substance of interest must be determined and documented. The assay must be able to discriminate
between compounds of closely related structures."

In this case, it specifically and legally is their job to prove a negative. Or, as stated positively in the ISL, to prove that they have detected "only the substance of interest." They did not do so.

syi

m said...

Mike Solberg,

I think you misinterpret Ayotte's testimony. She is not admitting that any of that stuff would be a violation of the IS, only that it would impact the accuracy of the measurements and would have to be taken into account. She was under cross-examination, and Jacobs was asking leading questions. So when he asked "they would still have to show they properly identified the substance", she was not interpreting that as a legal question but a scientific one.

Legally, it is still Landis' burden to show that any of that stuff violated the IS, and it is not the labs duty to exclude every contaminants\ by producing the mass spectra.

This from the majority opinion:

"Dr. Ayotte confirms this statement in noting
that a laboratory does not violate Article 5.4.4.2.1 of the ISL just because it
produces a chromatogram that contains matrix interference. Therefore, even where matrix interference has occurred in the Stage 17 chromatograms it would
not amount to a violation of the ISL."

Matrix interference refers to other substances in the urine.

Also OMJ said there were partial mass spectra produced, and these were consistent with the 5B and 5A in those peaks. So the 5A and 5B were clearly identified in those peaks.

But not all the mass spectra were produced, so one can't exclude that some other substance might have been present. But he counted that as a very small probability, since that substance would have to elute at **exactly** the same time as the 5A and 5B and be hidden in their peaks, since there were no mini-peaks shown in the 5A and 5B. It was Landis' burden to propose some such special substance in his urine and to show that it might elute at exactly the same time as the 5A and 5B. Remember it couldn't be anything in normal urine. He never proposed cortisone or any other substance as co-eluting.

Larry said...

M -

I promised you a post on the Brenna testimony, and I'm working on it as time allows. But in the process of writing that post, I decided that it would be better to focus on a different issue first.

I've argued that there was a departure from the ISL, since the evidence shows that LNDD did not have criteria in place for the identification of IRMS compounds satisfying the requirements of TD2003IDCR. You've made a number of objections to my argument, one of which is that my arguments are based on indirect and circumstantial evidence. In order to find an ISL departure, you want direct evidence, the "best" evidence (so to speak). You would have required the FL team to produce the LNDD's SOP, so we could see what's in it. You would have the FL team solicit testimony from LNDD witnesses on the contents of their SOP. You'd have the FL team ask the witnesses directly, did LNDD ever adopt TD2003IDCR criteria for inclusion in the SOP, and if so, what was the criteria?

Would that they could.

I point you to Procedural Order No. 2 in this case. The web cite to this order is lengthy - probably the quickest way to get hold of it is on TBV's page for 5/1/07, under "Hearing - Monday Documents." This is a document that has escaped my attention until now, and for that I'm truly sorry. There's a great deal about this case that becomes clear only after reading this document.

Procedural Order No. 2 provides that the FL team is not entitled to any documentation, other than the documents included in the so-called "lab pack". The SOP is not included in the lab pack, and if the LNDD had criteria for identifying compounds under TD2003IDCR, those criteria would not have been included in the lab pack, either. The lab's documentation of how it identified IRMS peaks was never disclosed to the FL team.

And it gets worse. I give you paragraphs 5a, 5b and 5c from Procedural Order No. 2:

a. A discovery response that there is no document will preclude the production of the document, along with any related testimony through a witness at the hearing.
b. A discovery response that there are documents that exist but they are not required to be produced because of the ISL or the other principles set out at paragraph 2 above will preclude the production of the document along with any related testimony through a witness at the hearing.
c. Any document used to determine the ISO certification or the WADA accreditation of the LNDD is not subject to discovery.

In other words, not only was the FL team denied access to the written criteria (if there was any) under TD2003IDCR, they were also prohibited from questioning witnesses about this criteria.

So M, there's our problem in a nutshell. The rules allow an athlete to prove an ISL departure. But where (as with TD2003IDCR) the ISL directs the lab to develop standards or criteria, the WADA arbitration rules do not permit the athlete to examine the standards or criteria, or even to ask questions about the contents of the standards or criteria.

That's one hell of a system, M. That's a Catch-22 worthy of Joseph Heller: you can prove an athlete innocent if you show that a lab failed to meet its standards, but you're not permitted to know what the standards are.

So, M: given the restrictions set forth in Catch 22 -- er, I mean, in Procedural Order No. 2 -- you and I are left with two options:

1. We can say, simply, that no athlete can ever prove a departure from the ISL provisions governing identification of substances, since no athlete will ever be permitted to learn the criteria that was used by the lab to identify substances. In other words, we can simply read TD2003IDCR out of the ISL altogether ... and live with the idea that a lab's identification of substances is presumed valid under all circumstances and regardless of the evidence. I will tell you, M, that I can't live with that. I would regard such a state of affairs as a complete denial of due process.

2. The alternative option is the one I've been pursuing, which is that we struggle along as best we can with whatever indirect and circumstantial evidence we can muster about what the lab's TD2003IDCR criteria might be, or for that matter, about whether the lab had any such criteria in place. I will admit to you, it is maddening to have to do this, knowing that in the briefcase of someone on USADA's side of the table was a lousy piece of paper that would have answered all of our questions if we were just allowed to see it ... or even to ask questions about it. But we can't see the piece of paper, and we can't ask questions about it. We can just try to infer its contents.

So M, let it be understood that my circumstantial and indirect evidence regarding LNDD's TD2003IDCR criteria IS the best evidence available ... not just the best evidence available in the arbitration record, but the best evidence that could be elicited under this lousy system by a talented legal team dedicated to discovering the truth notwithstanding the obstacles placed before them.

Luckily for me, this indirect and circumstantial evidence is strong enough to make my case.

In any event, the evidence I'm able to produce here concerning the LNDD policies should not be discounted based on some notion we both shared that better evidence could have been produced by the parties.

P.S.: Mike, I think you're right, I'll try to post something in your defense soon!

Larry said...

Mike S. and M -

I'll weigh in here, what the heck.

Mike S., M has a point here. But M, you're reading of the ISL is way too narrow IMHO.

ISL 5.4.4 is about the methods that WADA labs are supposed to develop to detect doping compounds. The labs are supposed to develop, validate and document these methods, to show that the methods "are fit for the purpose" and meet the other standards set forth in the ISL. So, when ISL 5.4.4.2.2 states that "the ability of the assay to deterct only the substance of interest must be determined and documented", I think the "documentation" required is for the method as a general matter. I don't think that 5.4.4.2.2 requires that every lab doping analysis must include such documentation.

So on this point, for the moment, I agree with M.

However, M, there's more to the story. The ISL in 5.4.4.2.2 requires that the LNDD's methods must meet a specificity requirement. In other words, the LNDD's method for detecting testosterone in its IRMS runs must be able to "detect only the substance of interest." This is a requirement for the method, I agree -- it's not necessarily a requirement for every test run using that method. However, the ISL does require that such a method be developed and that it be used. If the LNDD does not have such a method in place, or if it failed to follow the method, we have an ISL departure.

So, we're back to our usual ISL analysis, beginning with: did LNDD have a method in its SOP to achieve specificity? If so, what was the method, and did it follow the method in the FL case?

If Mike S. can show that the only such method available to LNDD was the mass spectra, and that LNDD did not utilize the mass spectra to achieve specificity, then I think he's got his ISL departure.

Mike Solberg said...

m, I must be misreading you, because it sounds like you are saying that it is okay that the test is inaccurate, as long as it is not an ISL violation. Huh?

Also, I think you mischaracterize the quote from Ayotte. The majority (in paragraph 240) interpreted that quote to be talking about the limits of the word "should" rather than "shall" or "must" in talking about matrix interference. Matrix interference and specificity are different. The specificity bullet contains "must" language.

Matrix interference refers to the noise at (and below) the baseline (stuff that has "slipped through" and isn't really eluting at its proper time), while specificity specifically refers to "compounds of closely related structures." The way the GC separation works (it is, of course, a physical process, dependent on the structure of the molecule), compounds of closely related structures could very well elute at the same time. Thus the need to have the specificity bullet point in the ISL.

It is also interesting to me that the majority did not attempt to say that 5.4.4.2.1 only applies to the method, not actual lab practice. Again, their rejection of an ISL violation of 5.4.4.2.1 referred to the lack of "shall" language in the "matrix interference" bullet, not the inapplicability of the standard to actual lab practice.

And, m, I sincerely thank you for participating at TbV like this. It is much more interesting discussion when someone is legitimately and skillfully arguing the other side.

syi

m said...

Larry,

Brief response to your procedural order #2 post.

SOP is not the same as "identification criteria". I believe they were entitled to discover this.

They were not prevented from asking the witnesses about the identification criteria, and in fact they did ask Mongongu and Brenna about this.

Landis' attorneys apparently agreed to the limitations in the procedural order.

Besides I think I have solved the mystery about what criteria was used. They used TD2003IDCR as I will outline in a later post.

m said...

1) What criteria did the lab purport to use to identify the metabolites that were tested? 2) What criteria did the lab actually use?

The short answer to 1) is that they purported to use TD2003IDCR including the 1% standard. The short answer to 2) is that they mostly used TD2003IDCR including the 1% standard.

The following is an outline of my argument with references and fleshing out to follow:

1. Factual Background: The lab performs a GCMS on a reference material containing all 4 of the metabolites to be identified, here Cal Mix and blank urine, and also a GCMS on the Landis sample. Retention times, spectral and other data are measured for these GCs and compared with the corresponding RTs in the Landis samples. If they match an identification is found. Next the lab performs a GC-IRMS on the same Landis samples, the same blank urine, and on a Cal Mix that contains 3 out of the 4 metabolites to be measured. In the GC-IRMS the substances are burned up, so spectral data which aids in identification is lost. The RTs are compared and an identification is made. The testimony is confusing as to this last step. Which RTs are being compared with which other RTs? Witnesses spoke of comparing RRTs as measured from an internal standard substance, and not comparing the RTs directly with the metabolites in the Cal Mix or blank urine.

2. TD2003IDCR says among other things that one way to identify a substance is to match its RT with the RT of that same substance known to be in a reference material, including a urine sample “spiked with that material”. It states that RTs must match within 1% although this limit can be relaxed.

3. USADA pre-hearing brief states (p. 23) that the lab used TD2003IDCR as their identification criteria, including the 1% standard, and that this applied to RRTs as well as RTs. When you look at the lab docs cited, (USADA 0149-51) it makes explicit that the lab was using this criteria, but only with respect to the GCMS, i.e. comparing the GCMS RTs with the Cal Mix RTs. The RTs matched. Since I couldn’t read the final briefs I wasn’t sure whether USADA had maintained this position after the hearing testimony.

4. The majority opinion states that the lab complied with the 1% standard for both GCMS and GC-IRMS taken individually, but not across machines as Meier was arguing.

““Furthermore, as Dr. Meier-Augenstein attested, the RTs measured for the
GC/MS instrument and the GC/C/IRMS instrument separately are within the
1% criteria. There is no dispute on this point between the parties.”

I could confirm this for the GCMS, and now I think I have been able to confirm this for the GC-IRMS. USADA 0360 and 0362 show the Cal Mix runs for the B sample and show RTs of: a) 870.6, b) 1241.8, c) 1316.7, d)1491.1. These RTs correspond in the blank urine and Landis sample to respectively: a) the 5A andro internal standard, b) the Etio in the F2 sample fraction, c) the 5B andro in the F3 sample, d) the 11 keto in the F1 sample. Further it appears from USADA 351 that the RTs for all four of the metabolites in the sample match the corresponding RTs for the blank urine. It appears that the lab might have been using the blank urine as its reference material, and treating it as a spiked urine sample. Similar documents exist for the IRMS on the A sample as well.

5. So it appears that the lab has in fact matched the RTs of the samples with the RTs of at least 3 out of the 4 metabolites in the reference material or Cal Mix. The only one not matched was the 5A andro. Since the positive result was for the 5B Andro, and the 11 Ketio, this is not fatal to USADA’s case.

6. So how reconcile all the testimony by Brenna, Ayotte, Mongongu, and even Meier about the use of RRTs measured from an internal standard. I don’t know. My guess is that the process described by Ayotte and Brenna is actually the way most labs do it. Even Meier didn’t say, look the proper way to do this is to compare the RTs of the GC-IRMS directly with the RTs of the Cal Mix. Instead he went off on a bogus attempt to compare the GCMS RTs with those of the GC-IRMS RTs. I really believe we’re missing something here because we are not scientists. None of the scientists either in the Landis case or on DP seem to have a problem with using RRTs or looking at the peaks. It’s only anal lawyers. The other possibility is that the ID procedure in TD2003IDCR is not thought by scientists to be applicable to IRMS, or not understood well. Labs have developed their own procedures, with only lip service to the TD2003IDCR. This notion is supported by the fact that a future IS entitled “Reporting Guidance for Gas Chromatography/Combustion Isotope Mass Spectrometry Reporting Guidance” is being planned.

Larry said...

M -

I appreciate the fact that you're carrying a heavy burden here, trying to deal with me and Mike S. at the same time, plus being about the only person on TBV willing to defend the FL decision. Like Mike S., I'm grateful for your presence here. Otherwise, Mike S. and I would be left "preaching to the choir", so to speak.

But I need you to take a more careful look at Procedural Order No. 2, and on the WADA rules that underline Procedural Order No. 2. This Order is not unique to the FL case; this order presents the "Catch-22" that every accused athlete must confront in a doping case. This order is not something that the FL team "agreed to" -- it is an order that was forced upon them.

I'll walk through the history that led to Procedural Order No. 2.

1. FL's document request to USADA is contained in a letter dated October 16, 2006 from Howard Jacobs. You can see this letter at http://trustbut.blogspot.com/2006/11/rejected-doc-request-correspondence.html. If you review this letter, you'll see that there was NOTHING that the FL team didn't ask for! The items that the Fl team asked for included the following:

- Any Standard Operating Procedure or SOP used by LNDD related to the processing of sample 995474 by GC-C-IRMS.

- All documents that evidence, reference or relate to the current IRMS criteria used by LNDD for determining an Adverse Analytical Finding.

- All documents that evidence, reference or relate to the identification of each of the peaks in the IRMS analysis of sample 995474.

- All mass spectral data necessary to identify all peaks within the MSD TIC analysis in connection with sample 995474.

- All data that has been used to identify the peaks in the IRMS analysis in connection with sample 995474, including any relevant isotope standards not provided within the laboratory documentation provided to date.

- All documents that evidence, reference or relate to the selection of metabolites used by LNDD for the carbon isotope ration test for testosterone using any GC-C-IRMS method.

- Please identify the precise time at which each peak on the MSD TIC scan appears.

I encourage you to read the Jacobs letter yourself. He asked for EVERYTHING, believe me. So we can both put aside the idea that somehow, the FL team did not ask to see the TD2003IDCR criteria.

2. USADA responded to the Jacobs request by letter dated November 3, 2006, as follows (see http://trustbut.blogspot.com/2006/11/no-documents-for-you-correspondence.html):

"After extensive review by us of your voluminous requests, I am writing to inform you that we will not be providing any documents or other information in response to your requests. As you should know, the rules applicable to this proceeding establish the set of documents that are provided by the laboratories when a sample tests positive. After studying your requests and those rules, every request you make appears to seek documents or information not called for by the rules."

In short: Jacobs asked for everything, and USADA said he'd get nothing. Nothing, that is, except for the documents in WADA's standard document package.

3. The arbitrators first got into the act on February 2, 2007, when they issued Procedural Order No. 1. In that order, the arbitrators deferred making any rulings on discovery matters:

"The Panel has also been advised that a disagreement has arisen between the parties concerning the discovery of documents. Counsels are to make further submissions on this matter after which the Panel will make its determination and any necessary orders."

4. We then reach Procedural Order No. 2, which I discussed in an earlier post. Your understanding of Procedural Order No. 2 seems to differ from mine, so let's look at it in depth. (http://www.usocpressbox.org/usoc/pressbox.nsf/6272c9a938d3a5cb8525711000564abd/9009e75c100f0679852572db006438fa/$FILE/Procedural%20Order%20No%202.pdf)

(a) Procedural Order No. 2 relies on WADA's Technical Document TD2003LDOC on Laboratory Documentation Packages. (See http://www.wada-ama.org/rtecontent/document/lab_docs_1_3.pdf) I won't quote from this TD, but will only point out that the TD does NOT require a lab to disclose any of the criteria it has developed under the ISL. The TD requires the lab to produce the documents we've all seen during the arbitration and on this site: collection control forms, internal chain of custody documentation, test results and the like.

(b) Under Section 7.1 of the ISL as well as the TD, the Laboratory is not required to provide any documentation not specifically included in the Laboratory Documentation Package. Section 7.1 of the ISL reads as follows:

"the Laboratory is not required to support an Adverse Analytical Finding by producing, either to the Testing Authority or in response to discovery requests related to the hearing, standard operating procedures, general quality management documents (e.g., ISO compliance documents) or any other documents not specifically required by Technical Document on Laboratory Documentation Packages. References in the International Standard for Laboratories to ISO requirements are for general quality control purposes only
and have no applicability to any adjudication of any specific Adverse Analytical Finding."

M, you stated that "SOP is not the same as 'identification criteria'. I believe they were entitled to discover this [identification criteria]." Sorry, but you're wrong on this point. It should be clear from the foregoing that the FL team was prohibited from discovering LNDD's documentation of its identification criteria.

(c) Procedural Order No. 2 added its own twist to the restrictions placed on the FL team by ISL 7.1 and TD2003LDOC. The WADA rules prohibit production of documents other than those listed in the TD -- the majority arbitrators went a step further and prohibited any testimony on the content of documents:

"a. A discovery response that there is no document will preclude the production of the document, along with any related testimony through a witness at the hearing.
b. A discovery response that there are documents that exist but they are not required to be produced because of the ISL or the other principles set out at paragraph 2 above will preclude the production of the document along with any related testimony through a witness at the hearing.
c. Any document used to determine the ISO certification or the WADA accreditation of the LNDD is not subject to discovery."

M, you stated that "They [the FL team] were not prevented from asking the witnesses about the identification criteria." Again M, I think you're wrong here. Procedural Order No. 2 states that the FL team was prohibited from seeing the written identification criteria and from seeking "related testimony" concerning the criteria. The phrase "related testimony" could not be much broader! How could they ask witnesses about the lab's identification criteria if they were barred from seeking testimony "related" to this identification criteria?

M, you stated that "in fact they [the FL team] did ask Mongongu and Brenna about this [the identification criteria]." M, I say this with all due respect: you're wrong on this point. Please revisit the cross-examination of Brenna in Brenna's first round of testimony -- the FL team never asked him about the LNDD's identification criteria. Why? Because they were prohibited from doing so under Procedural Order No. 2. Please revisit the cross-examination of Mongongu. Suh was able to ask Mongongu what steps she took, when she took those steps and how long it took to perform each step. He did not ask her anything about the criteria she used. why? Because he was prohibited from doing so.

Now ... you raise the point that, according to Procedural Order No. 2, the prohibitions on testimony described above were contained in "agreements reached covering the documentary discovery process." You infer from this quote that the FL team agreed to these restrictions. However, there's no indication in Procedural Order No. 2 that the FL team was a "party" to these agreements. The order simply says that agreements were reached. Perhaps the agreements were among the arbitrators. It's simply not clear from the Order.

In any event, I'm not sure why it would be relevant to our inquiry whether these restrictions were 100% forced on the FL team, or whether the FL team had any ability to shape these restrictions. The point is, the restrictions were there. They were quite real.

FL's team could not get written copies of the LNDD criteria.

FL's team could not ask questions about the content of the LNDD criteria.

FL's team was forced to do what I've been doing (and interestingly, what you've done in your latest post on the TD2003IDCR criteria): they were forced to infer the contents of the TD2003ICDR criteria: by asking the LNDD witnesses what steps they performed to identify the IRMS peaks, by asking the USADA experts how they would go about identifying IRMS peaks in this case, and by looking at the documentary package to see how LNDD appeared to identify the IRMS peaks. Under the circumstances, this was the only way open to them. And unless we're simply going to say that a lab's identification of IRMS peaks is presumed valid under all circumstances and regardless of the evidence, this is the only way open to us.

Larry said...

M -

Regarding your post claiming that LNDD actually and successfully used the TD2003IDCR standard ... wow. From my non-scientific background, I don't see how that can be. Dr. M-A and Dr. Brenna missed that? All the science types here and on DP missed that? Even OMJ would have missed that. I guess it's possible ...

I have not focused on how LNDD identified peaks in the GC/MS test, but I always assumed that they used RTs against a reference sample run contemporaneously on the GC/MS. I don't think there was any controversy about this, and I think this would explain the USADA brief on RTs and the statement in the majority opinion that the GC/MS RTs matched within 1%. In any event, I'm not trying to argue here that there was anything wrong with the GC/MS identification.

My understanding (and again, I'm no scientist) is that the LNDD did not identify the FL IRMS peaks by comparing them to a reference sample run contemporaneously with the FL sample. I'll put aside all of the testimony from Mongongu, Ayotte, et. al. to the effect that the IRMS peaks were identified by comparing them to the GC/MS peaks. You've raised this issue yourself in your point 6 -- I'll just point out the following:

- if LNDD was actually identifying IRMS peaks by comparing them to the peaks shown in a contemporaneous IRMS reference sample, then there would have been no need to complete a GC/MS test as part of the IRMS procedure.

- my understanding (based on the Mongongu testimony) is that LNDD does run a reference sample (I believe this is the mix cal acetate) through the IRMS, but this is for the purpose of calibrating the instrument (see testimopny p. 430, pdf p. 308). I suppose that the results of a calibration run, but that does not appear to be the purpose behind LNDD's use of the mix cal acetate.

- my understanding is that the results of the mix cal acetate run are shown in USADA exhibit 361, which you can find quickly at http://trustbut.blogspot.com/2007/09/retention-times-ii.html. According to TBV, the mix cal acetate run contained the 5aA internal reference and one of the three metabolites, 5bA. The mix cal run used by LNDD was missing the other two metabolites, 5aA and 5bP. Interestingly, if I'm reading the test results correctly, FL's sample was negative for 5bA. The positive finding, I think, was based on the metabolites not contained in the mix cal acetate.

I'll leave the rest to the science types here to figure out. My only additional comment is in reference to your comment about "anal lawyers." Aren't you GLAD we don't have any of THOSE kind of lawyers around here?

m said...

Correction to my last post.

The 5A Androstandial was found positive in the IRMS test, not the 5B Androstandial. My mistake. Can't keep them all straight.

DBrower said...

Let's move to a new post for comments.