Picking up the conversation from comments in Larry: ISL Violations any way you look, we take the last few comments and start again.
We too would like to thank M for his diligence in posting and continuing the discussion. It wouldn't be anywhere near as revealing and interesting if he weren't around.
m said, (with clarification via email later)...
What Identification Criteria were used in the IRMS test?
1) What criteria did the lab purport to use to identify the metabolites that were tested? 2) What criteria did the lab actually use?
The short answer to 1) is that USADA Pre-Hearing brief claims that the lab used the identification criteria of TD2003IDCR including the 1% standard. The short answer to 2) is that they mostly used TD2003IDCR either including the 1% standard, or using a more relaxed standard.
1. Factual Background: The lab performs a GCMS on a reference material containing all 4 of the metabolites to be identified, here Cal Mix and blank urine, and also a GCMS on the Landis sample. Retention times, spectral and other data are measured for these GCs and compared with the corresponding RTs in the Landis samples. If they match an identification is found. Next the lab performs a GC-IRMS on the same Landis samples, the same blank urine, and on a Cal Mix that contains 3 out of the 4 metabolites to be measured. In the GC-IRMS the substances are burned up, so spectral data which aids in identification is lost. At this point the RTs may be compared and an identification can be made. The testimony and record are confusing as to this last step. Which RTs are being compared with which other RTs? Meier spoke of comparing the RTs of the GCMS against those of the GC-IRMS. Brenna, Ayotte, and Mongongu spoke of comparing the relative retention times (RRTs) as measured from an internal standard substance, and not comparing the RTs directly with the metabolites in the Cal Mix or blank urine. The majority decision says the RTs were compared directly to the reference substances and matched.
2. TD2003IDCR says among other things that one way to identify a substance is to match its RT with the RT of that same substance known to be in a reference material, or a urine sample “spiked” with that subtancel. It states that RTs must match within 1% although this limit can be relaxed, and it appears that this applies to RRTs also.
3. The USADA Pre-Hearing Brief stated (p. 23) that the lab used TD2003IDCR as their identification criteria, including the 1% standard, and that this applied to RRTs as well as RTs. When you look at the lab docs cited, (USADA 0149-51) it makes explicit that the lab was using this criteria, with the assumption that the GCMS matching of RTs and RRTs was sufficient to satisfy the identification requirements of TD2003IDCR and validate the identification in the CM-IRMS. The RTs matched. Since I couldn’t read the final briefs I wasn’t sure whether USADA argued this position after the hearing testimony.
USADA’s Pre-Hearing brief stated:
“Once the retention times and ion ratios were all entered in the form, the LNDD criteria for compound identification were applied. The LNDD criteria for compound identification are the WADA criteria defined in the WADA Technical Document entitled Identification Criteria For Qualitative Assays Incorporating Chromatography and Mass
Spectrometry, TD2003IDCR (Exhibit 12, page 2), as shown on page USADA 0149 (page section immediately below sample number):
ToldrancesJix~eps ar 1 'AMA (document : WADA Technical Document - TD2003IDCR)Meaning:
Toldrances sur le tr et le trr : +/- 1 % ou +/-0.2 min (prendre le plus faible des deux)
Pour les abondances relatives >50% il est admis +/-I 0% (en absolu) de variation 25 <>
Tolerance defined by WADA (document: WADA Technical Document TD2003IDCR)
Tolerance for the RT and RRT: +I- 1 % or +I-0.2 min (whichever is smaller)
Relative Abundance >50% : acceptable +/-I 0% variation (absolute)
25 to 50% +I-20% (relative)
“The retention time or relative retention time of the compound in the sample must fall within the range defined by the retention time or relative retention time (respectively) of the reference compound, plus or minus the required value. The ion ratio for each ion in the compound in the sample must fall within the range defined by the corresponding ion ratio for the reference compound, plus or minus the required value. The evaluation of all data against criteria was all confirmed and documented on USADA 0149-0151 (Exhibit 24). Thus, LNDD identified the six compounds in Respondent's sample according to WADA TD2003IDCR.”
(USADA pre-hearing brief at 23-25)
USADA’s brief points to USADA 0149 as a summary of the identification data and criteria. USADA 0149 states in French that the document contains the GCMS qualitative analysis for the GC-IRMS confirmation, and that TD2003IDCR constitutes the identification criteria. This document indicates that the matching of the RTs and RRTs in GCMS against the reference material, Mix Cal Acetate, also constitutes the TD2003IDCR identification data for the GC-IRMS. That is, there is no direct matching of the GC-IRMS RTs or RRTs against the reference substances in the Cal Mix. If that is the proper interpretation, then one can argue that this identification procedure complies with TD2003IDCR when we relax the 1% requirement.
4. The majority opinion stated that the lab complied with the TD2003IDCR 1% standard for both the GCMS and GC-IRMS taken individually, but not across machines as Meier was arguing.
““Furthermore, as Dr. Meier-Augenstein attested, the RTs measured for the
GC/MS instrument and the GC/C/IRMS instrument separately are within the
1% criteria. There is no dispute on this point between the parties.”
I could confirm this for the GCMS, and now I think I have been able to confirm this for the GC-IRMS. USADA 0360 and 0362 show the GC-IRMS Mix Cal runs for the B sample and show RTs of: a) 870.6, b) 1241.8, c) 1316.7, d)1491.1. These RTs correspond to in the blank urine and Landis sample respectively: a) the 5A andro internal standard, b) the Etio in the F2 sample fraction, c) the 5B Andro metabolite in the F3 sample, d) the 11 keto in the F1 sample. So the Cal Mix contained a metabolite from each sample fraction, F1, F2, F3, but not the 5A Andro metabolite in F3.
Further it appears from USADA 351 that the RTs for all four of the metabolites in the sample match the corresponding RTs for the blank urine. It can be argued that the blank urine can be treated as a spiked urine sample and qualifies as a reference material for purposes of compliance with TD2003IDCR, although it does not appear that the lab treated it as such.
Similar documents exist for the IRMS on the A sample as well.
5. It appears that the lab has in fact matched the RTs of the samples with the RTs of at least 3 out of the 4 metabolites in the reference material or Mix Cal. The 5A Andro was not included in the Mix Cal so was not matched. The 5A andro is the metabolite that tested positive for exogenous testosterone. Nevertheless, there can be no doubt that the 5A andro was properly identified. The Mix Cal properly identified the 5B Andro in Landis’ F3 sample, and we know from the GCMS and the GC-IRMS urine blank that the 5A Andro directly follows the 5B by about 30 seconds. In the Landis F3 sample, the 5A Andro peak also directly follows (with no intervening peaks) the 5B by about 30 seconds.
6. So how reconcile all the testimony by Brenna, Ayotte, Mongongu, and even Meier about the use of RRTs measured from an internal standard. I don’t know. My guess is that the process described by Ayotte and Brenna is actually the way most labs do it. Even Meier didn’t say, look the proper way to do this is to compare the RTs of the GC-IRMS directly with the RTs of the Cal Mix. Instead he assumed the proper method was to compare the GCMS RTs with those of the GC-IRMS RTs. I really believe we’re missing something here because we are not scientists. None of the scientists either in the Landis case or on DP seem to have a problem with using RRTs in the GCMS and then matching those peaks in the GC-IRMS. It’s only anal lawyers. The other possibility is that the ID procedure in TD2003IDCR is not thought by scientists to be applicable to IRMS, or not understood well. Labs have developed their own procedures, with only lip service to the TD2003IDCR. This notion is supported by the fact that a future IS entitled “Reporting Guidance for Gas Chromatography/Combustion Isotope Mass Spectrometry Reporting Guidance” is being planned.
7. Conclusion: The retention times and relative retention times in the GC-IRMS matched those in the blank urine, which may be considered a spiked urine sample within the meaning of TD2003IDCR, and substantially matched those in the Mix Cal, which can be considered a reference material within the meaning of TD2003IDCR. Alternately, or in addition, the relative retention times of the GCMS matched those in the GC-IRMS in compliance with TD2003IDCR when we relax the 1% matching requirement. In all these cases, the identification criteria was TD2003IDCR itself.
Original at 11:47 AM, but modified via email later.
I appreciate the fact that you're carrying a heavy burden here, trying to deal with me and Mike S. at the same time, plus being about the only person on TBV willing to defend the FL decision. Like Mike S., I'm grateful for your presence here. Otherwise, Mike S. and I would be left "preaching to the choir", so to speak.
But I need you to take a more careful look at Procedural Order No. 2, and on the WADA rules that underline Procedural Order No. 2. This Order is not unique to the FL case; this order presents the "Catch-22" that every accused athlete must confront in a doping case. This order is not something that the FL team "agreed to" -- it is an order that was forced upon them.
I'll walk through the history that led to Procedural Order No. 2.
1. FL's document request to USADA is contained in a letter dated October 16, 2006 from Howard Jacobs. You can see this letter at http://trustbut.blogspot.com/2006/11/rejected-doc-request-correspondence.html. If you review this letter, you'll see that there was NOTHING that the FL team didn't ask for! The items that the Fl team asked for included the following:
- Any Standard Operating Procedure or SOP used by LNDD related to the processing of sample 995474 by GC-C-IRMS.
- All documents that evidence, reference or relate to the current IRMS criteria used by LNDD for determining an Adverse Analytical Finding.
- All documents that evidence, reference or relate to the identification of each of the peaks in the IRMS analysis of sample 995474.
- All mass spectral data necessary to identify all peaks within the MSD TIC analysis in connection with sample 995474.
- All data that has been used to identify the peaks in the IRMS analysis in connection with sample 995474, including any relevant isotope standards not provided within the laboratory documentation provided to date.
- All documents that evidence, reference or relate to the selection of metabolites used by LNDD for the carbon isotope ration test for testosterone using any GC-C-IRMS method.
- Please identify the precise time at which each peak on the MSD TIC scan appears.
I encourage you to read the Jacobs letter yourself. He asked for EVERYTHING, believe me. So we can both put aside the idea that somehow, the FL team did not ask to see the TD2003IDCR criteria.
2. USADA responded to the Jacobs request by letter dated November 3, 2006, as follows (see http://trustbut.blogspot.com/2006/11/no-documents-for-you-correspondence.html):
"After extensive review by us of your voluminous requests, I am writing to inform you that we will not be providing any documents or other information in response to your requests. As you should know, the rules applicable to this proceeding establish the set of documents that are provided by the laboratories when a sample tests positive. After studying your requests and those rules, every request you make appears to seek documents or information not called for by the rules."
In short: Jacobs asked for everything, and USADA said he'd get nothing. Nothing, that is, except for the documents in WADA's standard document package.
3. The arbitrators first got into the act on February 2, 2007, when they issued Procedural Order No. 1. In that order, the arbitrators deferred making any rulings on discovery matters:
"The Panel has also been advised that a disagreement has arisen between the parties concerning the discovery of documents. Counsels are to make further submissions on this matter after which the Panel will make its determination and any necessary orders."
4. We then reach Procedural Order No. 2, which I discussed in an earlier post. Your understanding of Procedural Order No. 2 seems to differ from mine, so let's look at it in depth. (http://www.usocpressbox.org/usoc/pressbox.nsf/6272c9a938d3a5cb8525711000564abd/9009e75c100f0679852572db006438fa/$FILE/Procedural%20Order%20No%202.pdf)
(a) Procedural Order No. 2 relies on WADA's Technical Document TD2003LDOC on Laboratory Documentation Packages. (See http://www.wada-ama.org/rtecontent/document/lab_docs_1_3.pdf) I won't quote from this TD, but will only point out that the TD does NOT require a lab to disclose any of the criteria it has developed under the ISL. The TD requires the lab to produce the documents we've all seen during the arbitration and on this site: collection control forms, internal chain of custody documentation, test results and the like.
(b) Under Section 7.1 of the ISL as well as the TD, the Laboratory is not required to provide any documentation not specifically included in the Laboratory Documentation Package. Section 7.1 of the ISL reads as follows:
"the Laboratory is not required to support an Adverse Analytical Finding by producing, either to the Testing Authority or in response to discovery requests related to the hearing, standard operating procedures, general quality management documents (e.g., ISO compliance documents) or any other documents not specifically required by Technical Document on Laboratory Documentation Packages. References in the International Standard for Laboratories to ISO requirements are for general quality control purposes only
and have no applicability to any adjudication of any specific Adverse Analytical Finding."
M, you stated that "SOP is not the same as 'identification criteria'. I believe they were entitled to discover this [identification criteria]." Sorry, but you're wrong on this point. It should be clear from the foregoing that the FL team was prohibited from discovering LNDD's documentation of its identification criteria.
(c) Procedural Order No. 2 added its own twist to the restrictions placed on the FL team by ISL 7.1 and TD2003LDOC. The WADA rules prohibit production of documents other than those listed in the TD -- the majority arbitrators went a step further and prohibited any testimony on the content of documents:
"a. A discovery response that there is no document will preclude the production of the document, along with any related testimony through a witness at the hearing.
b. A discovery response that there are documents that exist but they are not required to be produced because of the ISL or the other principles set out at paragraph 2 above will preclude the production of the document along with any related testimony through a witness at the hearing.
c. Any document used to determine the ISO certification or the WADA accreditation of the LNDD is not subject to discovery."
M, you stated that "They [the FL team] were not prevented from asking the witnesses about the identification criteria." Again M, I think you're wrong here. Procedural Order No. 2 states that the FL team was prohibited from seeing the written identification criteria and from seeking "related testimony" concerning the criteria. The phrase "related testimony" could not be much broader! How could they ask witnesses about the lab's identification criteria if they were barred from seeking testimony "related" to this identification criteria?
M, you stated that "in fact they [the FL team] did ask Mongongu and Brenna about this [the identification criteria]." M, I say this with all due respect: you're wrong on this point. Please revisit the cross-examination of Brenna in Brenna's first round of testimony -- the FL team never asked him about the LNDD's identification criteria. Why? Because they were prohibited from doing so under Procedural Order No. 2. Please revisit the cross-examination of Mongongu. Suh was able to ask Mongongu what steps she took, when she took those steps and how long it took to perform each step. He did not ask her anything about the criteria she used. why? Because he was prohibited from doing so.
Now ... you raise the point that, according to Procedural Order No. 2, the prohibitions on testimony described above were contained in "agreements reached covering the documentary discovery process." You infer from this quote that the FL team agreed to these restrictions. However, there's no indication in Procedural Order No. 2 that the FL team was a "party" to these agreements. The order simply says that agreements were reached. Perhaps the agreements were among the arbitrators. It's simply not clear from the Order.
In any event, I'm not sure why it would be relevant to our inquiry whether these restrictions were 100% forced on the FL team, or whether the FL team had any ability to shape these restrictions. The point is, the restrictions were there. They were quite real.
FL's team could not get written copies of the LNDD criteria.
FL's team could not ask questions about the content of the LNDD criteria.
FL's team was forced to do what I've been doing (and interestingly, what you've done in your latest post on the TD2003IDCR criteria): they were forced to infer the contents of the TD2003ICDR criteria: by asking the LNDD witnesses what steps they performed to identify the IRMS peaks, by asking the USADA experts how they would go about identifying IRMS peaks in this case, and by looking at the documentary package to see how LNDD appeared to identify the IRMS peaks. Under the circumstances, this was the only way open to them. And unless we're simply going to say that a lab's identification of IRMS peaks is presumed valid under all circumstances and regardless of the evidence, this is the only way open to us.
Regarding your post claiming that LNDD actually and successfully used the TD2003IDCR standard ... wow. From my non-scientific background, I don't see how that can be. Dr. M-A and Dr. Brenna missed that? All the science types here and on DP missed that? Even OMJ would have missed that. I guess it's possible ...
I have not focused on how LNDD identified peaks in the GC/MS test, but I always assumed that they used RTs against a reference sample run contemporaneously on the GC/MS. I don't think there was any controversy about this, and I think this would explain the USADA brief on RTs and the statement in the majority opinion that the GC/MS RTs matched within 1%. In any event, I'm not trying to argue here that there was anything wrong with the GC/MS identification.
My understanding (and again, I'm no scientist) is that the LNDD did not identify the FL IRMS peaks by comparing them to a reference sample run contemporaneously with the FL sample. I'll put aside all of the testimony from Mongongu, Ayotte, et. al. to the effect that the IRMS peaks were identified by comparing them to the GC/MS peaks. You've raised this issue yourself in your point 6 -- I'll just point out the following:
- if LNDD was actually identifying IRMS peaks by comparing them to the peaks shown in a contemporaneous IRMS reference sample, then there would have been no need to complete a GC/MS test as part of the IRMS procedure.
- my understanding (based on the Mongongu testimony) is that LNDD does run a reference sample (I believe this is the mix cal acetate) through the IRMS, but this is for the purpose of calibrating the instrument (see testimopny p. 430, pdf p. 308). I suppose that the results of a calibration run, but that does not appear to be the purpose behind LNDD's use of the mix cal acetate.
- my understanding is that the results of the mix cal acetate run are shown in USADA exhibit 361, which you can find quickly at http://trustbut.blogspot.com/2007/09/retention-times-ii.html. According to TBV, the mix cal acetate run contained the 5aA internal reference and one of the three metabolites, 5bA. The mix cal run used by LNDD was missing the other two metabolites, 5aA and 5bP. Interestingly, if I'm reading the test results correctly, FL's sample was negative for 5bA. The positive finding, I think, was based on the metabolites not contained in the mix cal acetate.
I'll leave the rest to the science types here to figure out. My only additional comment is in reference to your comment about "anal lawyers." Aren't you GLAD we don't have any of THOSE kind of lawyers around here?
Correction to my last post.
The 5A Androstandial was found positive in the IRMS test, not the 5B Androstandial. My mistake. Can't keep them all straight.
[BACK FROM MAIN BODY]