Tuesday, October 23, 2007

Commenter "M": Majority was correct

Landis Case Review by Commenter "M"


1. Introduction
2. The Arbitration Decision
3. Burden of Proof
4. T/E Ratio Test
5. IRMS Test
6. Amory testimony
7. Concluding Remarks

1. Introduction

I was invited by Bill Hue to present the case supporting the majority decision. Unfortunately, I can only go into a few of the issues raised by the parties. As TBV has observed, it appears that the key legal issue is whether the IRMS retention times violated the applicable technical standards.

I was trained as a lawyer, although I haven’t practiced for a long time. I also have training in statistics and economics, but no biological sciences training whatsoever. So understanding the science here was difficult and possibly imperfect. I read large portions of the transcript (but not all), some of the legal briefs that where available, and tried to understand the exhibits. But I certainly might have missed something.

[MORE]


2. The Arbitration Panel’s Finding of Doping.

USADA charged Landis with using a prohibited substance, artificial testosterone, based on 2 test results from his stage 17 urine samples: 1) elevated testosterone to epitestosterone (T/E) ratios, and 2) IRMS tests showing artificial testosterone in his urine. Additionally, IRMS retests of Landis B samples from other stages were positive for artificial testosterone as well, even though no elevated T/E ratios had been found in the initial A samples. The IRMS test is a direct test, it can determine whether the testosterone is artificial. The T/E ratio test is indirect; elevated T/E ratios most likely result from doping, but could occur from other causes. The majority decision sustained the doping charge on the basis of the IRMS test results on stage 17, but found that the T/E test results were defective because the lab failed to “monitor” 3 ions as was required by one of the technical standards (IS). The majority found some other lab errors including at least one IS violation, but concluded that none of these impacted the IRMS test results.

The dissent agreed that the T/E tests violated the IS, but also found that the IRMS test was defective because of testimony that the T/E ratios and certain metabolytes did not behave as predicted by scientific studies, and because some labs used a more stringent criteria for finding exogenous testosterone. These defects however did not violate any technical standard. The dissent also argued that various and cumulative lab record keeping and other errors discredited all the test results including the IRMS test results.


3. Burden of Proof and Requirements for Finding a Doping Violation

A doping violation may be proved by any credible means including confessions or admissions, testimony, or more relevantly lab testing results. The anti-doping organization (here the US Cycling Federation and USADA, or “USADA” for short) has the overall burden of proof to show a doping violation to the “comfortable satisfaction” of the arbitrators. This is more than a “balance of the probabilities” (analogous to preponderance of the evidence in US civil trials), but less than beyond a reasonable doubt as in US criminal trials.

What do these differing burdens of proof mean? Some people have claimed that they cannot feel comfortable convicting Landis unless they are 99% to 100% sure. This is clearly not realistic or appropriate. We know that scientific tests have error; they give false positives and false negatives. I’ve read that doping tests give many more false negatives than positives. That’s why so many confessed dopers have never tested positive. In court cases, proof by a preponderance of the evidence the standard used in civil trials technically means only a 51% probability of assurance, although studies have shown that judges usually require something more like 55%, and jurors require even more. Proof beyond a reasonable doubt for conviction of a crime usually requires around 80% assurance by jurors up to a high of 95% for murder. So proof by a “comfortable satisfaction” of a doping violation should probably only require 70% to 80% assurance if that.

(See, Handbook of Jury Research)

3A. Presumption that scientific test results valid


Where a finding of doping is based on lab tests, both the A sample and B sample must test positive. In such cases, USADA is aided by the legal presumption that the WADA labs have conducted their scientific analyses in compliance with scientific standards, as laid out in the WADA International Standard for Laboratory Analysis (ISL), that is, the scientific results are valid. The athlete may rebut this presumption by showing a departure from a WADA international technical standard (IS). The athlete need only show such a departure by a “balance of the probabilities” (preponderance of the evidence). If such a departure is shown then the burden shifts back to USADA to prove that the departure did not “cause” the doping finding. It is unclear to me whether the burden of proof on USADA in this regard is the “balance of probabilities” or a “comfortable satisfaction”, but the majority decision seems to have adapted the “balance of probabilities” standard. If USADA can’t prove that the departure did not cause the doping finding, they lose.

The key point I would point out here is that the departure from an International Standard should be causally connected to the finding of doping, otherwise the doping finding should stand. As a general proposition the violation of technical rules or sloppy lab procedures should not invalidate the test results unless causally connected to those test results. I think this is a reasonable rule. We should not let off an otherwise guilty athlete as a prod to improve unrelated sloppy lab work. The remedy for sloppy lab work is the yearly testing during accreditation process or other administrative penalties, not letting off guilty athletes. The Landis majority decision seems to adopt this view in stating that the arbitration panel must “weigh the evidence” to determine whether an IS violation “affected” the finding of doping:

“Therefore, violations of the ISO 17025 or of WADA Technical Documents can be violations of the ISL for purposes of rebutting the initial presumption favouring the Lab that an AAF has been established. However, that of itself does not mean that the AAF does not amount to an anti-doping rule violation. The Panel must weigh the evidence to determine if the violation affected the AAF. If that is the case then the anti-doping rule violation may not have been made out at law.” (emphasis added)

If an IS violation is established it is not clear what sort of proof is required to show that the violation did not cause the finding of doping. In the Landaluce case before CAS, the majority decision found a violation of the IS mandating that the same person cannot perform the A sample test and the confirmatory B test. In that case the person performing the IRMS confirmation test, had done some minor work on the A test. This was said to be due to workload and understaffing. On its face this violation would seem to have little factual causal connection with the IRMS doping finding. It has more to do with the appearance of impartiality. The majority also found that UCI failed to show that this violation did not cause doping finding. The UCI presented no evidence to show that this violation did not cause the doping finding, but just argued that it didn’t. So it’s unclear what showing would be required. Is it sufficient to show that the B tests were performed up to standard and that there was no evidence of bias or impropriety? Or must one completely throw out the B and even the A tests because the same person worked on both tests, and prove the doping by other means. In nearly all cases this would be impossible. I would argue that the former is appropriate. The latter would let off an otherwise proven doper on a technicality and would undermine the confidence of non-doping athletes in the fairness of the system. It would be an incentive for others to dope to keep up with the dopers.

Assuming no IS violation is shown, what weight is to be given to other scientific evidence aimed at discrediting the doping finding? Landis presented evidence of other lab errors, evidence of alternative laboratory practices, alternative more stringent positivity criteria, and general scientific evidence suggesting that the tests results were inconsistent with doping (the Amory testimony). The WADA rules appear to bar the use of such evidence to rebut the presumption and the majority agreed. Article 18 reads:

“Compliance with the International Standards (as opposed to other alternative standards, practice or procedure) shall be sufficient to conclude that the procedures addressed by the International Standards were performed properly.”

The majority decision stated that an IS violation “is the only relevant evidence to determine if the Athlete’s attempt to rebut the presumption of Article 18 may be successful. Proving some other procedure, practice or alternative standard is of no consequence in rebutting the presumption favouring the Lab.”

On the other hand, assuming an IS violation is shown by the athlete, is other general scientific evidence not constituting an IS violation now generally admissible and relevant for the general purpose of discrediting the test methodology and results? The rules on this don’t seem to be clear. If USADA only presents evidence to show that the IS violation was not causally connected with the doping finding (as it might have in the Landaluce case), then I would argue that the scientific evidence should be limited to that issue. If USADA tries to show the doping violation by rehabilitating the test results or by other test results, then the permitted scientific evidence should be expanded to address those issues, and this might include general scientific challenges like the Amory testimony. So whether, and for what purpose, this additional evidence may be relevant will depend on the facts of the case. This appears to be an open question, and its possible a panel might permit general scientific evidence to attack the overall validity of the testing results. Campbell in his dissent seem to treat this type of evidence as relevant to discrediting the IRMS results even though he found no specific IS violation with respect to those results. I think he was wrong to do so.



3B. Are the WADA procedures unfair?

The arbitration rules and procedures reflect a compromise between cost, accuracy and speedy resolution on the one hand, and an athlete’s ability to contest a finding on the other. I don’t think that that line was drawn unreasonably. WADA scientists have developed scientific tests for the presence of doping compounds that take into account cost and the probability balance between false negatives and false positives. The WADA detailed technical standards reportedly resulted from the legalistic advocacy of USADA which sought to impose uniformity. I wonder whether this is an overly legalistic approach. Its possible that science involves too much variety and moves too fast for codified standards to give the answer in all cases.

The presumption of scientific validity is reasonable and removes the need to litigate the scientific validity of a test in every arbitration, which would be expensive and wasteful. There are limitations on the right to discovery in this arbitration, but the arbitrators seem to have the discretion to order additional discovery. This is typical of many if not most arbitrations. Again I think this is a reasonable limitation in the interest of expediting the hearing process. Discovery as a matter of right in civil court proceedings can be incredibly time consuming and costly. It is also possible that revealing too much scientific information publicly, e.g. the background testing and validation behind the tests, could aid dopers in devising ways around the test. Most employees fired for violating a drug policy, don’t enjoy anywhere near the contractual due process rights that the athletes do here. I do find the rigidity of the WADA strict liability rules at times unreasonable, and I think more discretion should be given in this respect. I’ve read that the WADA code was heavily influenced by the USADA’s legalistic, rule bound, and inflexible approach to achieve uniformity.
­­

4. The T/E Test Results

The T/E tests were thrown out because of an IS violation, i.e. three ions were not monitored in the confirmation tests as required by TD2003IDCR. I don’t really disagree with this, so the crucial issue will come down to the validity of the IRMS results discussed below. I did go through the T/E arguments and summarize them here for my own understanding. You can skip this if not interested.

The T/E ratios on the B samples were found to be 11 to 1, and those on the A sample 6 to 1. Both exceeded the 4 to 1 threshold for positivity, however only one ion was monitored. The majority decision (and dissent) found an IS violation of TD2003IDCR because the lab did not monitor 3 ions for identification purposes in their confirmation tests of the T/E ratios. The lab monitored only 1 ion, the m/z 432 ion, in both its screening tests and confirmation tests. USADA failed to show that this violation did not result in the doping finding, and accordingly the majority threw out the T/E results.

TD2003IDCR governs the identification of compounds and reads:

“The laboratory must establish criteria for identification of a compound. Examples of acceptable criteria are:
.....................................................
“In some cases it may be necessary to monitor selected ions to detect the substance at the Minimum Required Performance Limits. When selected ions are monitored, at least three diagnostic ions must be acquired.”


On the other hand TD2004EAAS governs testosterone testing and defines a threshold T/E ratio as follows:

The T/E value is given by the peak area or peak height ratio of testosterone and epitestosterone ......obtained by measuring the ion at m/z 432 by GC/MS analysis in a Single Ion Monitoring mode (SIM)..... The confirmation of the identity of any steroid reported with abnormal properties must be made (refer to technical document TD2003IDCR).”


TD2004EAAS specifies a specific ion, m/z 432, for testosterone testing, but only requires that 1 ion to be tested. Since this standard refers to TD2003IDCR for confirmation tests, the majority found that 3 ions were required for those tests. Was it “necessary” to monitor multiple “selected ions” in the confirmation tests? The majority “interprets” TD2003IDCR to mean this, but it’s not apparent to me on its face that that is the only reading. TD2003IDCR states that “in some cases it may be necessary” to use multiple “selected ions”. What was the evidence that this was such a case? The majority stated that if only 1 ion was required that the technical standards should have specified this with more “precision”. In requiring greater precision for the confirmation test, the majority decision was persuaded by Dr. Goldberger’s testimony that ion m/z 432 was contained in at least 10 substances other than testosterone and that it was “normal” practice to monitor 3 ions. I didn’t read any contrary testimony. USADA apparently argued that the lab’s single ion positivity criteria was documented for ISO inspection, and implicitly approved by ISO and since the ISL incorporated ISO standards, this approval meant that the labs positivity criteria met ISL standards. The majority’s reading took the position that the specific (IS) controlled over the general ISO certification.

Three ions were reportedly “acquired” in the tests but only 1 was “monitored” or analyzed. I wasn’t able to find any explanation from the testimony about why only 1 ion was monitored. Did the testers fail to follow known lab procedures, or did the lab and/or testers interpret the standards to only require 1 ion? I wasn’t able to find this out, but maybe I missed something. Campbell in his dissent rhetorically implies that if the lab couldn’t get the T/E test right, this is evidence that they couldn’t get the IRMS tests right. I think this is a pretty weak evidence entitled to little or no weight, since there is no causal nexus between the missing ion results and the validity of the IRMS tests. The IRMS results must be judged on their own.


5. The IRMS Test Results

The majority’s decision is based on the IRMS results showing artificial (“exogenous”) testosterone in both the A and B samples of Landis’ stage 17 urine sample. In addition, IRMS results showing exogenous testosterone in Landis B samples in other stages is offered as corroborating evidence. GC-IRMS testing isolates certain metabolites of testosterone, and then tests their carbon 13 to carbon 12 ratio to determine whether the source of the testosterone is artificial or natural (“endogenous”). If the measured ratio is sufficiently different, here “3 deltas”, from the ratio for natural testosterone, then a doping violation is found.

There seem to be 3 major challenges to these results: A) the metabolites were not correctly identified because retention times did not meet standards, B) the carbon ratios of those peaks were not correctly measured, C) other labs require the carbon ratios of at least 2 metabolites to exceed the 3 delta threshold.


5A. Retention times violated TD2003IDCR

Landis argued an IS violation in that the retention times (RTs) and relative retention times (RRTs) of the metabolites as measured by the GC-IRMS didn’t match closely enough to the respective RTs and RRTs measured by the GCMS as specified by the international standard TD2003IDCR, e.g. within .2 minutes or 1%. Consequently they claim that the metabolites were not adequately identified and any IRMS calculated carbon ratios were invalid.


TD2003IDCR reads in relevant part:

“The Laboratory must establish criteria for identification of a compound. Examples of acceptable criteria are:

Chromatographic separation

For capillary gas chromatography, the retention time (RT) of the analyte shall not differ by more than one percent or +/- 0.2 minutes (whichever is smaller) from that of the same substance in a spiked urine sample, Reference Collection sample, or Reference Material analyzed contemporaneously. In those cases where shifts in retention can be explained, for example by sample overload, the retention time criteria may be relaxed.”


Retention time (RT) is the time it takes for a specific analyte/metabolite to travel through the gas chromatographic (GC) apparatus until it is detected. RT can be measured from various starting points including the retention time of another known substance (“internal standard”, “chromatographic reference”). Relative retention time (RRT) is the ratio of the retention time of the analyte to the retention time of some other substance such as the internal standard. Different metabolites will have different RTs and different RRTs so in principal they can be distinguished and identified based on their RTs, although some different substances may have the same RTs making identification more difficult.

In general retention times for the same analyte performed on the same machine will vary depending on various GC factors, including the column, flow rate, column pressure, carrier gas, temperature, and dead volume. (Shimadzu website) So these factors need to be made and remain constant on the same machine for the retention times to always match within the limits. Retention times may also vary on different machines of the same type because the GC factors cannot always be made identical. Clearly retention times will also vary for different types of machines.

Landis’ expert Meier-Augenstein (“Meier”), testified that the retention times (RTs) of metabolites measured in the GCMS and the RTs of the same metabolites measured in the GC-IRMS varied by up to 8 minutes, and the RRTs varied by up to 7%. Meier appeared to measure the metabolite RTs from the 5a Androstanol, although I’m not absolutely sure about this. It’s not clear from the transcript how Meier calculated his RRTs. For example, did he adjust for the longer combustion time of the GC-IRMS. Landis argued that both the .2 minute standard and the 1% standard were violated, however Meier did concede that as between two different machines only RRT should be used as a basis for comparison. Meier also testified that he might expect RTs for the same analyte to vary by 1 to 2 minutes between different GC machines in his lab.

Both the majority and minority decisions take the position, based on the testimony of Brenna, that the 1% standard doesn’t apply to GC identifications made on different types of machines as in this case, but only to GC identifications made on the same machine. So there was no IS violation. The majority states that the RTs and RRTs when measured on each machine individually met the .2 minute and 1% standard.

Brenna testified that the IRMS process involves an additional combustion and drying processes which adds to the RTs of the analytes. He stated that accordingly neither the straight RTs nor RRTs could be compared between GCMS and GCIRMS. The majority decision characterized this additional time as “constant”. Some have argued that if that is the case, then in principal one can subtract out that constant time and thus make the RTs and RRTs comparable. However, there is little evidence on this issue and the majority decision seems to have mischaracterized it. Meier said that one must compare RTTs but said nothing about the time added by IRMS, or how such added time would affect the calculation of the RRT. He does not say whether he corrected for this additional time in calculating his RRTs. I wonder whether he wasn’t being deceptive by this omission. Brenna said that the added time would be “approximately” the same.

Brenna:

A,......“Because in the GC, molecules move through the GC
at a rate which is characteristic of each individual molecule, so a molecule that moves through the GC slowly will move through it at
that rate, and compared to a molecule that moves through the GC more quickly. So, and then when it emerges into this region here, all the molecules move through this region at approximately the same rate. So there is what we call "differential retention" here, but not here, and that has implications for calculating retention times and also relative retention times.”

“Q. So, for example, in your laboratory, would you expect the retention times for your GC to correspond with -- your GC/MS to correspond with your GC IRMS?
A. No. And we run GC/MS every day, and GC/C-IRMS every day, and we match our peaks 25 every day.
Q. And how about relative retention times?
A. No, for the reasons I've outlined.”


According to the testimony of Brenna (and Ayotte if I recall), identification of the substance can be achieved by matching their peaks against the known chromatographic standard which is identified by it’s retention time, as well as comparing those peaks to the peaks in the GCMS of the identical sample. Retention times may be used to aid in this matching, but don’t have to match within the 1% standard. So far, I have read no testimony or other commentary that explains why an accurate identification cannot be made on this basis. When I, with my untrained eye, look at the peaks of the respective chromatographs they match and map onto on another despite their RRTs being off by 4%. Violation of the 1% standard, even if applicable, doesn’t seem to prevent identification in this particular case. I suspect that the arbitrators were similarly persuaded. Seemingly knowledgeable scientific posters on Daily Peleton, especially “onemintjulep” and “rational head” explain why the visual match is clear. So far nobody has been able to examine the peak correspondence and explain specifically why those peaks don’t match. Meier did testify that in the general case one could have visually identical chromatographs where completely different substances were being measured. But in this case the same sample, with the same substances, was run through the two machines. So all major peaks had to be accounted for. The only way to account for all major peaks was to map the analytes in the GC-IRMS onto those identified in the GCMS. There appear to be no alternative mappings possible.

I did want to examine some additional arguments about the retention time arguments since these may come up in the CAS trial.

Several things stand out in reading TD2003IDCR.

a. Arguably the lab can establish other criteria than those specified in TD2003IDCR.

The 1% retention time standard is an “example” of “acceptable criteria”. . However, at least with respect to the three ion requirement, the majority read TC2003IDCR to be mandatory where it was found to apply, and to exclude alternative standards. I think this was a generally reasonable result in this case, since USADA did not really offer scientific evidence justifying an alternative “one ion” standard. However, I think it must be clearly shown by scientific evidence that the technical standard applies to facts of the case. In this regard, I think the arbitrators must rely on scientific testimony as to the meaning of the standard, and should not rely on their own “interpretation” of the language. I would have like to have seen testimony from those who drafted the technical standards whether the three ion requirement applied here.

In the case of the 1% standard I would like to see testimony that it was intended to even apply to GC-IRMS testing, before claiming it was mandatory. IRMS testing on a separate machine seems to present special problems for using retention times. I found an 1998 Olympic technical committee draft of the 1% standard and it was limited to specific substances not including those tested here. This raises the question whether the 1% standard evolved before IRMS testing.

b. The 1% standard may not apply to relative retention times RRT.

The 1% standard applies by its terms only to “retention time” (RT), a direct measurement, but not to “relative retention time” (RRT), a ratio. The majority decision seems to assume, at least for the sake of discussion, that it applies to “relative retention times” also. Again, how can arbitrators decide whether the term “retention time” incorporates “relative retention time” without scientific testimony. How can we assume that the specific 1% standard, as applied to retention time, should also be applicable to relative retention times without scientific testimony. The Shimadzu website explanations of relative retention time, that I’ve read, state that the error in RRT increases for an analyte that elutes farther from the reference standard.. This suggests that an accuracy standard applicable to straight retention times may not be applicable to relative retention time.

c. Even Meier in his own lab could not meet the 1% standard.

The 1% standard is not absolute; it may be relaxed where shifts in retention time can be explained. In this case not only were different types of GC procedures used, GCMS and GCIRMS, but some of the GC conditions were probably not identical, in particular temperature. So even if the 1% standard might in some cases apply to RRTs between machines, this is a case where the 1% criteria should be relaxed.

Meier’s testimony suggests that even using the exact same GC conditions it would be impossible to meet the 1% standard for relative retention times between different machines whether of the same type or completely different types as in the Landis case.

Meier:
A. “....You use -- you use a retention comparison because that is usually a bit difficult because, at the best of times, no two GCs and no two identical columns, even if they're the same manufacturer, will give you identical retention times. You go for what's called relative retention times. You add an internal standard to which you relate the retention time of everything else.” (T – 1362)

“A. We've got Hewlett Packard's trace gas, Agilent, I think we even have an old 7 Varian. Probably, four -- four or five different types of GCs.

Q. Okay. And if I took an internal reference compound like 5-alpha androsterone --did I get that right that time -- 5-alpha androstenol AC, and I ran it in two of your different GC instruments, would you expect me to get the exact same retention time?

A. Not the exact same retention time, no. But if you used the same temperature break and the same helium flow, the same column, or, at least, let's say, because you can't have the same column in two instruments at the same time, so you're using the same column time from the same manufacturer, even ideally from the same batch, you should, within reason, such as, for instance, the plus or minus of one or two minutes, you should get the same retention time, yes”. (T- 1503)



Clearly “one or two minutes” exceeds the .2 minute standard. Some people have claimed that labs routinely can achieve retention time precision in the thousandths of seconds. This is clearly not possible in the type of chromatography conducted here.

And if you add two minutes or 120 seconds to the RT of 866 seconds for the 5A Andro internal standard and to the RT of one of the metabolites, your RRT could be off by as much as approximately 2 to 3%, again exceeding the 1% standard. Thus, even Meier in his own lab probably could not meet the 1% standard. It really appears that the 1% standard is too stringent for comparison of retention times across different machines. Even, Meier seems to concede that the standard can be and should be relaxed.

Q. It's supposed to be not more than plus or minus one percent?

A. It's not supposed to be -- I mean, in duplicate cases I think there's a bit of leeway. I have no idea what the leeway is, but I can't imagine it's 600 percent. Because from one percent to 6 percent, that's -- well, 500 percent difference. I can't -- I can't see that. So I have no confidence in the data in terms of peak identification whatsoever. (T – 1409-10)

5B. Carbon Ratios not measured correctly.

Landis claimed that the IRMS measurements were not accurate because of linearity and other problems, and because of manual adjustment of peaks. The majority decision discussed these in detail and rejected these arguments, and the dissent didn’t really discuss the scientific issues at. I don’t have time now to discuss these arguments in detail.

5C. Other labs require at least 2 metabolites to test positive

Landis argued that the lab should require positivity for all four of the metabolites tested, and the dissent argued that the lab should require positivity for two metabolites as is done in the UCLA WADA lab. However, the language of TD2004EAAS does not appear to require positivity for multiple metabolites. “The results will be reported as consistent with the administration of a steroid when the 13C/12C value measured for the metabolite(s) differs significantly i.e. by 3 delta units or more from that of the urinary reference steroid chosen.” Landis argued that positivity for only one metabolite was shown although this is disputed as explained below. The majority did not appear to rule on this question explicitly. It simply found that the IRMS results were valid. So it’s decision could be interpreted as requiring only 1 metabolite positive, or interpreted as finding that 2 metabolites had been positive. The dissent on the other hand claimed that the more stringent UCLA standard discredited the IRMS test results.

The IRMS test showed 6+ delta units for the 5Alfa diol- pdial pair in both the A and B samples, and 3.51 delta for the Andro 11 ketol pair in the B sample and 3.99 delta units in the A sample. The dissent accepted the argument that the 3.51 delta did not exceed the 3 delta threshold when one took into account the lab’s measurement error allowance of .8 units. While the lab as a matter of procedure appeared to use the error allowance, it does not appear that it was legally required to do so by the ISL. If it had not done so, the 3.51 delta would have met the threshold and two metabolites would have been positive in the B sample. Moreover, the 3.99 from the stage 17 A sample did pass the 3 delta threshold even if one took into account the .8 unit error. This along with the multiple positive delta results on the B samples from other stages leaves little doubt in my mind that positivity was shown, even if technically only one metabolite in the stage 17 B sample may have exceeded the delta threshold. Clearly requiring 2 metabolites, or all 4 metabolites as the Landis team argued, to test positive would result in fewer false positives. But it would also result in many many more false negatives. No scientific testimony was presented to show why using only one metabolite was not an adequate standard. In any case, WADA rules state that compliance with a more stringent alternative standard was not required.

6. Amory Testimony

Campbell in his dissent argues that the testimony of Amory raises sufficient doubts in his mind that he cannot be sufficiently confident in the IRMS test results. Since the testimony of Amory shows no IS violation in connection with the IRMS test, this was not sufficient legal grounds for rebutting the presumption that the IRMS test results were valid. This is why I think Campbell’s reasoning was contrary to the law, and this is probably why the majority decision did not even discuss the Amory testimony. Nevertheless, I want to examine the persuasiveness of the Amory testimony since much has been made of it.

Amory testified based on a review of the literature that the IRMS testosterone components, in particular the 5a-diol and 5b-diol,always moved in tandem, and further that above normal T/E ratios persisted over time especially if there was repeated doping. According to Amory, on stage 17 the 5a-diol tested positive, but the 5b-diol tested negative, and thus were not close enough to one another in value as predicted by the studies he had reviewed. The Landis T/E ratios did not test high in the stages other than stage 17. Amory argued that these facts were inconsistent with Landis having ingested artificial testosterone. Based on this testimony, Campbell concluded: “When you consider all the errors and ISL violations in this case, the fact that the results also do not comport with known science is dispositive. I cannot be comfortable satisfied that LNDD’s results are correct.”

However, there was other testimony that was inconsistent with Amory. There was testimony that micro-dosing and oral doping as opposed to injections would not lead to the persistence of above normal T/E ratios. Amory conceded these points but then commented that if the dosages were that low there would be no benefit to the athlete. This was humorous, since he had previously testified that testosterone couldn’t help a cyclist during the race even if administered at high doses. There was testimony that T/E levels fell during the course of a long race, so taking low doses of testosterone might just maintain one’s T/E levels.

Shackleton, (the leading expert on testosterone metabolism, but a bumbling witness) testified that theoretically it would not always be the case that the 5a and 5b diols would move in the same direction as claimed by Amory when someone had ingested testosterone. Several case study examples were introduced showing 5a and 5b diol values similar to Landis by subjects who had ingested testosterone. Landis claimed that these studies had not been refereed and thus should not be admitted, but this fact did not mean that the data was unreliable, only that the case study may not have been of sufficiently wide interest so as to justify publication in a journal. Moreover, the refereed articles quoted by Amory appeared themselves to be in the nature of case studies and thus anecdotal, although involving many more subjects over time. Amory offered little or no theory to explain why the components must always move together over time. Both Shackleton and Amory stated that the metabolic breakdown of steroid components by the body was an incredibly complex process. So given this, Amory’s claim was weak, not at all conclusive, and overstated.



[BACK FROM MAIN BODY]

7. Concluding Notes

From the evidence in the record, I believe the majority’s finding that the 1% retention time standard set out in TD2003IDCR was not applicable to the GC-IRMS was correct in this case. Accordingly, the GC-IRMS tests should legally be presumed to be valid. If Landis is able to present specific evidence that other labs use something like the 1% standard or even some relaxed standard in their GC-IRMS analyses, or specific evidence explaining why the visual peaks don’t match, then I might revise my opinion. That would be something he might want to attempt in the CAS appeal.

Whether or not any of the test results should have been thrown out on technical legal grounds, I came away from reading the evidence with something like an 80% subjective assurance that Landis had doped. The IRMS tests for the other stages were persuasive in this regard. Given that, I would not want Landis to get off on a technical violation of some legal rule.

The majority decision discussed in great detail many of the key scientific and legal issues. I wasn’t able to go through each of those issues in detail or to determine whether all of the legal questions were decided correctly, but it does appear that the majority acted diligently and conscientiously in going over the evidence. I was less impressed with the dissent. Campbell really did not address the scientific evidence in any detail, and much of his argument was rhetorical and ignored the legal rules he was supposedly interpreting. Perhaps this reflected his difficulty in understanding the scientific evidence. He claimed various procedural and rule violations but did a very poor job of showing any causal nexus with the testing results. The very first argument in his dissent claims bad faith cherry picking of evidence by the lab and documentation violations, but as to the cherry picking he doesn’t cite to the transcript so I couldn’t figure what he was talking about. To my mind the strongest arguments questioning the lab results were made in the doubts expressed by the majority decision, not the arguments raised by the dissent. Campbell was clearly biased in favor of the athlete, and I will take it as a given that at least one of the other arbitrators was biased in favor of USADA. In this regard, I can understand an arbitrator not wanting Landis to get off on some legal technicality, when he might have the subjective conviction that Landis doped. But my analysis above has been based on what was expressed in the written decisions and my attempt to understand the evidence.

57 comments:

Unknown said...

M,

You blew it when you brought the following two people into the argument - especially “onemintjulep” and “rational head” explain why the visual match is clear. OMJ and RH are relentless in their belief the labs should be allowed to make mistakes yet still ruing a man's career/life.

Notice how quiet they've been on the Mayo situation???

Also, i see way too much use of the words Assume, Perhaps, and most likely scare me.

And finally, you state the following: As a general proposition the violation of technical rules or sloppy lab procedures should not invalidate the test results unless causally connected to those test results. I think this is a reasonable rule. We should not let off an otherwise guilty athlete as a prod to improve unrelated sloppy lab work. The remedy for sloppy lab work is the yearly testing during accreditation process or other administrative penalties, not letting off guilty athletes.

HOW DO YOU KNOW THE SLOPPY LAB WORK DIDN"T CAUSE THE POSITIVE RESULT??

WHAT IF YOUR REPUTATION AND CAREER WHERE ON THE LINE?? Would you accept the above statement re: Sloppy lab work???

Larry said...

M -

Before I say anything else, BRAVO! I disagree with much of what you've written, but your work is impressive, well-reasoned, honest and comprehensive. I've intended to write something like this myself, and I still hope to do so, but you've set the bar very high here. Congratulations on a job well done.

I am hoping to engage you in some dialog regarding what you've written. My plan is to respond to the points you've written in more or less the order in which you've written them, in a series of relatively short posts (well, short for me!). This will hopefully allow for some back-and-forth. I will try to be frank and direct, but polite. Please read "IMHO" in between any two words I write to you, when you think it's needed to maintain the right kind of tone and dialog.

(Michael, I've read your post and I hope you'll participate in this discussion. But I think you should recognize that M has given us a detailed and thoughtful analysis. I think I'm representing a point of view similar to yours, but like M, I read OMJ with respect and admiration. I've addressed below some of the points you've raised in your post.)

Also, I plan to address the legal points head-on, and to address matters of public policy (i.e., whether the WADA rules are good rules) only when absolutely necessary for our legal discussion.

OK, with all this being said, let's move to the first point for discussion. I read your points 1, 2 and 3 as simply setting forth background information, so I'll move to your discussion in 3A.

Your discussion in 3A is critical, I think, to where the majority went wrong in the Landis (FL) case. You correctly point out that the athlete has the burden of proving a departure from the International Standard for laboratory analysis (ISL) published by WADA. Once proven, the burden shifts to the prosecuting ADA "to establish that such departures did not cause the Adverse Analytical Finding or the factual basis for the anti-doping rule violation." (World Anti-Doping Code 3.2.2)

M, you correctly point out that under this WADA rule, the ISL violation must be causally connected to the finding of doping. But I think you've made some critical mistakes in your analysis of this WADA rule.

The WADA rule is a bit broader than you described. You stated that the ISL departure must be causally connected to the finding of doping. But under the rule cited above, the causal connection can also be to the factual basis for the anti-doping violation. A small distinction, I'll grant you, but one that may prove important down the road.

The WADA rule places the burden of proof here on the ADA. If the athlete can establish an ISL departure, then it's up to the ADA to prove that the departure did not cause the finding of doping. If the ADA cannot meet this burden of proof, then it loses and the athlete wins. I don't think the WADA rules could be any clearer on this point.

In your analysis, you turn this burden of proof on its head (as did, I think, the majority opinion in the FL case). You state that any ISL departure "should be causally connected to the finding of doping, otherwise the doping finding should stand." IMHO, your analysis here is backwards. It is up to the ADA to prove that the ISL departure is NOT causally connected to the finding of doping, otherwise the doping finding must be overturned. If the ISL departure is of a type that makes it difficult to disprove a causal connection, then this is a problem for the ADA, not the athlete.

You appear to think that there are possible departures from the ISL that merely go to "violation of technical rules or sloppy lab procedures", and that these departures "should not invalidate the test results unless causally connected to the test results." This may be a defensible public policy opinion on your part, but it's not a correct reading of the WADA rule. The WADA rule ASSUMES that a departure from the ISL IS causally connected to the finding of doping, but allows the prosecuting ADA to rebut this assumption by proving that there was no such causal connection.

In this sense, WADA rule 3.2.2 is a mirror-image of WADA rule 3.2.1, which assumes that the lab has performed in compliance with the ISL in the absence of proof to the contrary from the athlete. We saw in the FL case that rule 3.2.1 is a heavy burden for the athlete to bear in these arbitrations -- even with all the proof offered by FL that the LNDD had screwed up multiple times, LNDD was STILL entitled to the presumption that it performed properly in every area where the FL team could not specifically prove otherwise. In light of the burden imposed on the athlete by rule 3.2.1, the burden imposed on the ADA by rule 3.2.2 does not seem at all unreasonable. Remember, the ADA has unlimited access to the lab in question to meet its burden of proof under 3.2.2, while the athlete's access to the same information is limited.

You also argue that a lab screw-up must be an ISL departure in order for that screw-up to be grounds for overturning a doping finding. This is a correct reading of the WADA rules. The lab can do any number of horrible things: delete data, manipulate data, fail to keep a decent record of the steps it took to perform a given analysis, but if these things are not addressed in the ISL, then they're not relevant to a WADA doping case. We saw a number of such screw-ups discussed in the FL decision, that the majority dismissed out of hand using this kind of reasoning. Given that a lab can get away with any number of screw-ups that are not addressed in the ISL, I have no problem holding a lab strictly accountable for its screw-ups that ARE addressed in the ISL.

You discussed the Landaluce case, and I think it is a perfect example to illustrate the point I'm trying to make. In the Landaluce case, the lab in question departed from the ISL by having the same person perform the A test and the B test. In such a case, the burden of proof falls to the prosecuting ADA to show that this departure did not cause the finding of doping or the factual basis for the finding of doping. You argue that this particular area of the ISL is aimed at "the appearance of impropriety" and is relatively unimportant. I would argue that this rule is concerned with the bias of the lab technician, and that this bias is critical to the validity of a WADA arbitration (just as the possible bias of the state's star witness in a criminal case is critical to the proper determination of the case). But it doesn't really matter what I believe or you believe. What matters is that the identity of the technician performing the A test and the B test is addressed in the ISL and is thus ASSUMED to be causally connected to the doping finding (that is, in the absence of ADA proof to the contrary).

You correctly point out that it would have been difficult for the ADA in the Landaluce case to prove that the ISL departure in that case was NOT causally connected to the doping finding. I agree. Perhaps the ADA could show that the B test in question was completely mechanical, and that it would be impossible for the bias of the lab technician to affect the results. Perhaps the ADA could prove that the lab technician in question was the athlete's grandmother, and was actually biased in favor of the athlete. Or perhaps there was nothing the ADA could do to win the Landaluce case, once this ISL departure was proven by the athlete. I don't know. All I know is that the WADA rules are clear on this point, and that the arbitrators (and on appeal, the CAS) are required to follow these rules.

M, as I said, this is a CRITICAL point. You and I have to be on the same page here, before we can look at the substantive matters at issue in the FL case. So I'll pause at this point and let you respond.

Larry said...

M, one further thing to point out: WADA rule 3.2.2 requires the prosecuting ADA to prove that any "departures" from the ISL did not cause the adverse finding. Note the use of the plural term "departures". In other words, the ADA's burden of proof is to show that all such departures, taken as a whole, did not cause the adverse finding. The ADA cannot try to pick off each single departure one by one. They must consider the cumulative impact of all such departures.

If you examine the decision of the majority arbitrators in the FL case, you'll see that they erred on this point. They considered the ISL violations singly and individually, not cumulatively and as a whole.

Unknown said...

M,
Thank you for your eloquently written and legally well reasoned article. You've helped me to see 1.) why the majority didn't address the Amory testimony and 2.) how the arbs, being people without scientific background, were persuaded by Brenna (and probably Botre) that Landis doped. However, as a scientific person with familiarity with GC principles, I still don't think he should have been convicted. You state "So far nobody has been able to examine the peak correspondence and explain specifically why those peaks don’t match." Visually, they look like they do match. While this might make you suspicious, it does not constitute positive identification by any recognized method that I'm aware of. It would have been astoundingly simple for LNDD to run the metabolites in the cal-mix (and they really needed to do this given the different temperature ramps used on the two machines). Due to either sloppiness or incompetence, they didn't do this. Consequently, pure and simple, they don't have positive identification at least for the S17 F3 we've been discussing). I relly believe that if the arbitration panel were composed of Analytical Chemists with GC backgrounds (with a Lawyer as the legal consultant), I have no doubt that Landis would have won.

Best

wschart said...

I have noticed something going on here as I read discussions like this and the Bustinbilly one a short while back. I'll read something like M's article here and think "Well s***, maybe the arbs were right." Since I am neither a lawyer nor a scientist, it's hard for me to rebut anything. Then someone like Larry (who is a lawyer) comes along with a rebuttal, and I'll think "Hey, maybe the arbs were wrong."

With that in mind, look at the process the panel used: first there was the hearing, USADA presented its case, Landis presented his, there was some rebuttal. We understand that after the public hearing, both parties had an opportunity to file briefs. The panel deliberated in some manner. But the panel also had Dr. Botre. What role he played is largely unknown. But if the panel was more influenced by the last opinion read or heard, could Botre's role here have been very influential?

bostonlondontokyo said...

"...the bias of the lab technician, and that this bias is critical to the validity of a WADA arbitration (just as the possible bias of the state's star witness in a criminal case is critical to the proper determination of the case)..."
Larry, I'm not sure I understand this. Yes, as a logical matter, of course one could say that having one's mom as a lab technician could affect results (if she decided to break rules in favour of her son) but I really, really, really think that you're stretching an already thin thread into something that once again sounds like 'conspiracy'. The comparison to a 'star witness' is not relevant here. We're talking about a lab technician who works her job, and deserves the basic right of believability (innocent before proven guilty...). No one has raised any argument to suggest that the lab tech had a personal grudge to harm Floyd Landis, nor has there been an argument raised to suggest that she was intimidated into 'creating' results by a higher-up. If Team Landis thought that this was the case, her background would have been dug up, questions raised during the arbitration, and her credibility as to bias would have been questioned in the proceedings. When you think of it that way, can you see how ridiculous a path that is? I feel that you are grabbing at straws when you start discussing the biases of lab techs - it's almost comical

Laura Challoner, DVM said...

M

Nice job! It isn't easy to put yourself out there for critical comment. Hopefully, you will continue to contribute. As you have seen, at this site, we believe ALL viewpoints should be intelligently debated in a civilized and respectful manner.

BLT,

Your problem is not with Larry, it is with the scientific method itaself and virtually all adjudicative bodies that require compliance with strict procedures for sample handling and testing before results are to be considered valid.

While "technical" matters like chain of custody, the requirement that 2 tests are necessary to confirm one another before validity exists and that one does not check one's own work when two results are required may seem ridiculous and comical to you (when styled as "bias") and overly "technical" to others, I can assure you that scientists and magistrates all over the world do not share that view. Those things form the foundation upon which conclusions are reached in science and in law. Deficits in foundation make whatever results obtained weak and/or invalid.

There are explicite ISL and WADA provisions for sample handling and testing. Whereas adjudicative bodies and the scientific method would invalidate results when those requirements are lacking, the WADA Code tries to over-rule the scientific method through the bizarre burden flips articulated in the WADA Code. Your view that such deficits don't or shouldn't matter unless actual "bias" or skullduggery of some sort are proven by the athlete is a position that is even rejected by WADA.

Under the WADA Code, the test is not whether the athlete can "prove" actual lab technician bias. The athlete simply must prove that the ISL requires two different technicians test the "A" and "B" sample. Then, where the world at large would simply dismiss the case or not admit this "evidence" because it lacks foundation, the WADA Code permits the ADA to rebut the presumption that the ISL violation didn't cause the adverse anaytical. How? Why? Through additional and perhaps at that point non-analytical proof that the athlete is a cheater nontheless.

The Landis majority was not brave enough or sure enough to get to that stage and finding. Instead, in my opinion, it tortured the ISL definitions to conclude there was no ISL violation, thus saving themselves from having to declare Landis a cheat upon the testimony of Papp, Lemond, his historic blood array or some other reason they were not confident enough in to endorse. Or, they might have faced the predicament they created when they permitted additional "B" sample tests and those samples to be adverse analytical findings themselves or additional evidence USADA presented overcoming the presumption created by the athlete's successful burden flip.

Larry said...

BostonLondonTokyo (BLT) -

Please read what I wrote in context. I said that from M's point of view, the identity of the lab technician is a matter of appearances. I said that from my point of view, the identity of the lab technician is a matter of substance. Then I said "it doesn't really matter what I believe or you believe." The fact that the ISL addresses the identity of the lab technician makes this identity a substantive issue in cases like the Landaluce case.

The WADA rules do not allow us to look at a departure from the ISL, and say, "who cares, we know that the athlete is really guilty". Instead, it's up to the prosecuting ADA to prove that the departure did not cause the adverse finding. In other words, the WADA rules ASSUME a causal connection between a departure from the ISL and the adverse finding, in the absence of proof from the prosecuting ADA to the contrary.

This is a critical point for all to understand. If you agree with what I've written so far, then it's not as important that you agree with what I've written below.

I introduced the topic of "bias" in an effort to explore how the prosecuting ADA might offer proof that a particular ISL departure did not cause an adverse finding. (In hindsight, I could have been clearer on this point.) The prosecuting ADA cannot meet this burden of proof without some discussion (or assumption) regarding why the ISL contained the rule that was violated by the lab. I should hesitate at this point before offering examples! But say that the ISL specified that all test tubes used by the lab must be clean, and say further that the athlete proves a departure from this specification. How could the prosecuting ADA prove that this departure did not cause the adverse finding? I think that any such proof would have to begin with an explanation of why it's important to use clean test tubes. So, the prosecuting ADA might suggest that test tubes are supposed to be clean in order to avoid the introduction of particular contaminants into a sample ... then they might offer proof that no such contaminants were present in the sample.

BLT, please don't get too hung up on the particulars of this example. Focus instead on the burden of proof placed under the WADA rules on the prosecuting ADA. This burden of proof effectively requires the ADA to tell us why the affected ISL provision was included in the ISL in the first place. Without this information, it's impossible to effectively examine the causal connection assumed to exist between the ISL departure and the adverse finding. Without this information, it's impossible for the prosecuting ADA to meet its burden of proof.

Now, you may disagree that the rule in the Landaluce case is present in the ISL to protect against bias. Fine. But if you're the prosecuting ADA, you're going to need to explain why the rule is there. I offered up "bias" as a possible answer, but ultimately it doesn't matter what I think!

Now, to clarify a few points. In my earlier post, I did not accuse any Landis lab technician of being biased. I agree that there's no proof of bias in the FL case. I brought up bias in the context of the Landaluce case, not the Landis case. This being said, I think that the ISL rule at issue in the Landaluce case addressed a human bias we all share: we don't like to admit that we're wrong. If a lab technician reaches a certain finding for an "A" sample, they're likely to want to reach the same finding for the "B" sample. That's the only bias directly at issue. (Again, I could have been clearer on this point.)

To make this point as clearly as I can: the arguments I'll make here regarding the FL case have nothing to do with the issue of "bias". I don't believe in conspiracy theories. I believe in correctly applying the rules to the facts of the case.

OK?

m said...

Larry,

Short response to your points, may not have more time today:

1. WADA 3.2.2 doesn't apply in this case, only WADA 3.2.1. WADA 3.2.2 mainly applies to when and how to take samples, and not the ISLs. So the "factual basis" language doesn't apply.

2. WADA 3.2.1 reads that if an ISL violation is shown, "then the anti-Doping Organization shall have the burden to establish that such a departure did not cause the Adverse Analytical Finding."

I am suggesting two things with respect to this:

A. Regardless of the burden, the panel on its own initiative may determine whether the violation is causally connected. The panel seems to take this position with its "weigh the evidence" statement. I may be putting too much on this language, but am comfortable with this reading for now.

In fact WADA 3.2.2, if it had been applicable here, would have supported this reading more than the language of 3.2.1. WADA 3.2.2 states that violations of standards "which did not cause an Adverse Analytical Finding or other Anti-doping rule violation shall not invalidate such results."

This means that not every violation rebuts the presumption only those which are "causally connected". But as I said, 3.2.2 isn't applicable here.


B. Leaving this aside, when USADA does have the burden, I don't see that much of a problem in proving that a violation of an IS is not causally connected, so long as USADA is not called on to prove the impossible.

3. The ruling in the Landaluce case was ambiguous, since the panel did not explain what sort of evidence might be needed to rebut the presumption. I suggested two extremes, a narrow view and a broad view. You side with a broader view. I side with a narrow view. I don't think this is a settled question. In any case, I think this is a factual question, and will turn on the particular IS that is claimed to have been violated each case. I will also add that I disagree with the reasoning in the Landaluce case, but accept that it may describe the applicable CAS law. (I haven't researched any other cases)

4. You say: "The WADA rule ASSUMES that a departure from the ISL IS causally connected to the finding of doping, but allows the prosecuting ADA to rebut this assumption by proving that there was no such causal connection."

I don't disagree with this statement. I'm just saying that there will be IS violations that on their face may be factually unlikely to be causally connected to the doping finding. In such cases, the burden on USADA will not be very great. For example, an IS violation wrt to the T/E ratios, has no causal connection with the IRMS tests. Even if USADA has the burden here, it can be easily proven. This "assumption" as you call it is just that, and can be rebutted by showing factual circumstances to the contrary.

m said...

DailyBob,

If matching retention times are the only way to achieve identification, can you explain why Meier himself says he wouldn't expect matches closer than "1 to 2" minutes even under ideal conditions in his own lab? Check out section 5.A comment c)

Larry said...

M -

What follows is a response to your last post.

On your point 1, you are right, I stand corrected. The applicable rule is 3.2.1.

On your point A, you'll need to explain further. Rule 3.2.1 places the burden on the prosecuting ADA to prove that a departure from the ISL did not cause the adverse finding. I see nothing in this rule or any other WADA rule that would allow the arbitration panel to supply this proof if the proof is not provided by the prosecuting ADA.

On your point B, I care not at all whether it's possible in a given case for the prosecuting ADA to rebut the presumption of a causal connection. Rule 3.2.1 places this burden on the prosecuting ADA without any exception for "impossibility". WADA reads its rules strictly against the athlete; it's fair to read these rules strictly against the ADAs.

Regarding paragraph 3, I don't think I have any quarrel with what you've written there. I think that the evidence required to rebut the presumption should follow the pattern I suggested earlier (figure out the reason for the presumed causal connection between the ISL rule and an adverse finding, then prove that the reason is not applicable to a given case), but I'm happy to live with the general idea that the proof required will depend on the facts of a given case.

Regarding your paragraph 4, again I have no problem in theory with what you've written.

So, if we can work out the differences represented by your paragaph A and my reaction to your paragraph A, I think we'll be on the same page and can move forward to the remainder of your Case Review.

wschart said...

OK, I've been thinking about the back and forth between M and Larry. It seems pretty clear to me, based both on the discussions above as well as the CAS decision in the Landaluze case, that once an IS violation is established, the burden is on the ADA to prove it did not produce the adverse finding, and is not on the athlete to prove that is did cause the AAF. UCI did not bother to attempt to prove that the same-tech violation did not cause the AAF for Landaluze, so CAS sided with him. Case dismissed.

The question has been raised "How can WADA/UCI prove a violation did not cause an AAF?" Opinion seems to be that this is very difficult to impossible. But here's an idea: if the lab has well documented all steps involved, including chain of custody issues as well as the methods used to maintain and set up equipment as well as the methods used to conduct the tests, the raw data obtained from all runs, and how these data were analyzed there would be a good case that the same results would have been obtained by another tech. Correct me if I am wrong, but weren't nos amis at LNDD involved with Landaluze too? And we know how their documentation is. Could this be why UCI folded in the Landaluze case?

Some posters here and elsewhere who are in the anti-Landis camp have implied that proper documentation is some kind of bureaucratic burden that dirty athletes attempt to use to "get off on a technicality". However, proper documentation can make the prosecution's case that much stronger. Imagine what the situation for Landis would be if LNDD had properly conducted and documented their tests? If USADA could have shown that everything was done properly, it would have been a slam dunk.

Larry said...

wschart, IMHO you have it exactly right.

Yes, there's a danger that under the WADA rules, the only way an athlete can win his case is on a "technicality". That's because the WADA rules are themselves highly technical. I think "M" has described this very well in his analysis.

Also agreed about the importance of documentation. Yes, as a general rule, good documentation should make for a stronger ADA case. But it's also true that without good lab documentation, it becomes more difficult for the athlete to make a substantive case. This is one reason why I think it's fair to throw out a doping case that has not been properly documented under WADA's own ISL rules.

In any event ... my focus here is on the WADA rules and on whether the majority arbitrators reached the right decision under those rules.

Unknown said...

M,
In 5A(c), M-A is talking about retention times between the two machines. What I've been trying to say is that the only reason that there is a discussion on retention times between the two machines is due to the fact that LNDD didn't have the other two metabolites in the cal-mix. This means the ONLY way they then had any hope of identifying them was using retention times (or RTTs) between the two machine, i.e., at that stage, it's a salvage operation. Then at a minimum, there would be a time offset due to combustion (in the IRMS) in addition to a slight shift due to other conditions such as vacuum, gas flow, etc.. This is the 1 - 2 minutes that Dr. M-A is referring to. The correct way to have done this would have been to run the reference standard and the 3 metabolites in the cal-mix, and then include the reference standard in the f3. This way, you could compare the rention times between the standard and the metabolites in the cal-mix and the f3. If they meet the 1% or 0.2 minute criteria ON THE SAME MACHINE, you have postive ID. I don't know any other way to do it and say the ID is positive. That they didn't do this (besides being bewildering) is what forced them to try and use retention times between the two machines. This was ultimately made impossible by the different temperature ramps. This is why I stated in a different thread that I view the s17 f3 chromatogram as being meaningless.

Larry said...

dailbob -

I've never been able to get this straight.

What you're saying is that LNDD ran a spiked sample through the GC/IRMS before they ran the FL sample, but the spiked sample only contained one of the three ions they needed to measure in order to test for endogenous testosterone? And presumably, the RT for this single ion DID match up with the corresponding ion in FL's sample within the criteria established in TD2003IDCR? But there was no reference run in the GC/IRMS for the other two ions? Making it necessary to try to identify these ions by reference to the GC/MS graphs?

Since we're trying to look at all this from the standpoint of the WADA rules, I'd like your take on how all this stacks up against the requirements of TD2003IDCR. First of all, TD2003IDCR requires that each lab must establish criteria for the identification of a compound. Did LNDD ever indicate what criteria it established here? And do you think the LNDD criteria meet the requirements under TD2003IDCR?

And forgive me if this is well-covered territory in other TbV posts. I DO have my own take on this, which I'll come to in due course in my discussions with "M", but I'd like to hear your take.

Unknown said...

Larry,

Check out this post by tbv for more detailed explanation to your question.

http://trustbut.blogspot.com/2007/10/our-appeal-brief.html

Cal

m said...

Larry,

Let's assume for the purpose of discussion that USADA has the burden to show that any IS violation did not cause the AFF.

BTW it's not absolutely clear to me that if they fail to show this, USADA necessarily loses. Based on a reading of WADA 3.2.1 alone, arguably USADA just lose the benefit of the presumption, and could theoretically prove the validity of the test results by scientific evidence to a "comfortable certainty". But I haven't researched this. Nobody wants to cross that bridge, and I won't either.

Larry said...

M -

No, disagree. Rule 3.2.1 states that if the athlete rebuts the presumption that the lab performed in accordance with the ISL, then the ADA has the burden of proving that the ISL departure did not cause the Adverse Analytical Finding.

IOW, the rule presumes that the ISL departure caused the Adverse Analytical Finding, and places the burden on the ADA to prove otherwise. If the ADA fails to meet this burden, then the arbitrators have to conclude that the adverse analytical finding was caused by the ISL departure. In such a case, the ADA loses.

M, I don't think there are any open questions here.

bostonlondontokyo said...

Larry, TbV: Thanks for responding to my post. In a re-read, yes I took the reference to 'bias' out of context (to some degree). I was generally responding to a slightly different take on the issue, and it doesn't cloud my reading of the discussion that is going on here.

I do want to point out, however, that although this thread has been relegated to a discussion of M's analysis of the majority verdict, I think it's important to keep one's senses grounded. I'm not a scientist, a judge or lawyer, and to my knowledge, I assume the purpose of this blog is to inform anyone who was interested in facts and observations about the Floyd Landis doping case. I enjoy following the threads - but remember that we all write (in a sense) as anonymous people. We can say whatever we'd like to about ourselves in posts, profiles, etc, but I have no idea of the credentials of anyone who is writing here, nor do I think that there is any valid way to prove credentials online - anything can be photoshop-ed or posted to a webpage that appears to be official. I write webpages professionally and am shocked with the means available to validate ANYTHING in a seemingly believable way nowadays.

In that sense I think it's worth remembering that all of us (who are probably non-professionals) ought to consider that any issue posted is up for discussion, dissemination. Fortunately, this is a good blog, and people do answer each others questions with respect.

To the issues of custody, the burden of proof, universal standards (if these even exist) I notice, as something of an observer here, that we lose the bigger picture in the minutiae. In the discussion of agreed upon standards, for example, the issue of 'clean testubes' was raised as a reasonable standard. What concerns me is that I'm feel that we are moving into frightfully uncertain ground - where does one draw the line for such standards? As a good example, has LNDD's water been tested to see if there are any trace minerals that could be left during drying of said test tubes that could affect a test result? Yes, it's funny to get that detailed in analysis, but then, it's not so funny.

SO - if the lab has not met the standards that many of you feel have been set as required standards, then the results should have been tossed. But they weren't. Now, we are discussing how the Arbs could have come to this conclusion despite what is seen as a slew of evidence saying that the results are not valid.

Shouldn't, then, the discussion be as follows: How is this information presented at the CAS hearing so that it IS taken seriously? We are WAYYYYY past the Arbs - they're history at this point - do we continue this as an academic discussion of reading a test result, or a way by which we view the available data in order to beef up the CAS case? I now B. Hue has cone much to create this document, but all the minds out there could probably come up with more?

I apologise for the very random post here, I just thought many issues were raised.

Larry / tbv, thanks for taking outsider points of view seriously, btw...

Larry said...

Cal & Lorie -

In order to properly respond to "M" on his point 5, I need some very basic information: I need to know:

1. Did LNDD have "established criteria" (within the meaning of TD2003IDCR) for identification of GC/IRMS compounds? Apparently, it did not, or at least no such criteria were presented during the trial. Instead, the arbitration examined evidence of the identification method used by LNDD in its practice.

2. Did LNDD run a spiked urine sample, reference collection sample or reference material sample through the GC/IRMS machine contemporaneously with its testing of the FL samples? On this point, all I can find in the most recent material posted by TbV is the following:

"It [LNDD] could have used a cal mix in the IRMS that had all the analytes of interest, and matched RT and/or RRTs on the same instrument. The LNDD chose not to do this."

This tells me what LNDD did NOT do; I need to understand what it DID do.

From my research, it appears that LNDD did perform a mix-cal acetate IRMS test "contemporaneously" (within 5 hours) of the IRMS test on FL's S17 sample. I assume that a "mix-cal acetate" is a spiked urine sample, a reference collection sample or a reference material sample (I'm not sure which kind of sample). But I gather that the mix-cal acetate only contained one of the three required ions? So that it could not be used to identify the IRMS compounds? So that the LNDD had to fall back on a visual comparison of GC/MS charts and GC/IRMS charts? Where is all this discussed?

Mike Solberg said...

I don't know if anybody is still paying attention to this comment thread, but I'll add my two bits.

First, M, great job. What a huge amount of work. It is one thing to read the relevant material and try to understand it. It is quite another to be able to put your analysis in writing. Impressive.

Next, as TbV has said, everything comes down to the IRMS, so that's what I'll address.

M, your analysis is of the IRMS results is right, up to a point, but I think you and the majority (and Campbell for that matter, as he was confused by M-A also) both succumbed to confusing testimony on both sides.

To wit, you say

"Both the majority and minority decisions take the position, based on the testimony of Brenna, that the 1% standard doesn’t apply to GC identifications made on different types of machines as in this case, but only to GC identifications made on the same machine. So there was no IS violation. The majority states that the RTs and RRTs when measured on each machine individually met the .2 minute and 1% standard."

In part, that is right. I agree that the .2 min. or 1% standard does not apply across machines. In that sense, there was no IS violation.

But the key question, then becomes "What you are left with?" How are you going to identify the IRMS peaks? How do you show that "This peak is 5A, and this peak is 5B"? And for that matter, how do you show "This peak contains ONLY 5A, and this peak contains ONLY 5B"?

There is nothing in the LDP which reveals how LNDD identified the peaks.

Brenna at first testified that you had to use retention times. Then, for some reason (possibly because USADA realized the importance of the fact that there was no reference material run in the IRMS - see below) Brenna later testified that you use the "visual matching" method. Brenna's first comments about this simply do not match his later comments.

In any case, there is nothing in TD2003IDCR about visual matching.

If LNDD was going to use some method other than the "examples" listed in TD2003IDCR, then they were required to document it (the first two lines of the TD). They did not do so.

I see it as a clear IS violation. They didn't follow the examples of acceptable procedure given, and they didn't explain and justify their own procedure. This IS violation should have sent the burden to USADA, making them prove that the violation did not cause the AAF (a burden which in my view they did not meet).

It is important to remember that paragraph you, M, quote about "Chromotographic separation" ASSUMES that the lab has run "a spiked urine sample, (or) Reference Collection sample, or Reference Material analyzed contemporaneously." They did not run any of those (on any of the runs we know of). The 1% or .2 min. standard applies to the difference between the analytes in any of those standards and the analytes in the target sample. This is the point dailbob has made repeatedly. They didn't run anything to compare the target analytes to.

Yes, they did run an internal standard for one metabolite. But what does that do for you? The only reasonable conclusion is that LNDD INTENDED to rely on a comparison of RRT's across machines. That is why M-A went to great lengths to show that, IF THAT WAS THE METHOD THEY WERE USING, then it didn't comport to TD2003IDCR.

Now, you say that no one has shown that the peaks aren't the right thing. But, as others have argued in this comment thread, that isn't the point. An IS violation did occur and the burden of proof is on USADA to show it didn't cause the AAF. They didn't do that.

Presumably the TD is there for a reason. There must be enough variability in IRMS peaks given different conditions that very specific protocols must be followed. LNDD didn't follow them, and they didn't provide any justification for what they did do.

syi

Mike Solberg said...

Larry, very good questions!

1) Correct. It did not. Dailbob's repeated point.

2) No, it did not. What did it do?, you ask. Good question. It never explained what it did. Brenna claimed after the fact (and contrary to his testimony the first time around) that LNDD relied on "visual matching." M-A (presumably not trusting "visual matching" - not even seeing it as a option, really) tried to figure out what they did do. The best he could come up with was that they relied on RRTs across machines. His point was that if that was what they did, it wasn't within the requirements of the TD. If they didn't do that, he had no idea what they did do - thus his dismissive attitude (see his testimony beginning on p. 1518 of the document - p. 1292 of the pdf).

Yes, they ran an internal standard for one metabolite on the IRMS. That's great for that metabolite. But it doesn't say much about the others, including the one which led to the AAF.

syi

DBrower said...

Larry,

Rats, I'd hoped that was clear, but evidently not.

In relative retention Figure 4 shows they ran a cal-mix GCMS on USADA 309 and USADA321. It has the 5aAC, 5bA, 5aA, and pdiol.

In retention times II, in Figure 5, USADA 361,345 and 349, we see tha cal-mix used in the IRMS has only the 5aAC and the 5bA, but not the 5aA and the Pdiol.

I think Mr. Idiot has a good explanation above.

TBV

Larry said...

TBV -

LOL! No, now that you point me to the right place, you could not have been any more clear.

So ... if all we're looking at is the issue of proper identification of the IRMS metabolites, arguably the only metabolite that LNDD was able to identify was the 5-beta-androsandiaol. I say "arguably" for two reasons. First, I'm not sure that the RTs for 5-bA for the mix-cal acetate and the FL sample actually match up within the tolerance specified under TD2003IDCR. (TbV, do you have any information on this?) Second, I suspect that "M" and I will need to argue about how TD2003IDCR should be applied.

Interesting. Going back to the results on FL's various tested samples, FL's S17 sample did NOT test positive on 5bA.

I may be a simple guy, but this looks like a slam dunk to me.

I'm curious, though. When the 7 other FL B samples were tested, what method did the testers use to identify the relevant metabolites?

And who is Mr. Idiot?

Mike, agreed 100%, you seem to see where I'm trying to go with my arguments.

DBrower said...

Larry,

Way back, Floyd was conversing with people on the Daily Peloton Forum. At some point, "Swim You Idiot" said some smart ass thing, and Floyd reacted in his "With all due respect, Mr. Idiot, ...", which name has stuck.

Mr. Idiot is Mike Solberg, but he will always go by his idiotic handle with us, with love and kindness.

The other samples were tested exactly the same way. There is no evidence that they have changed their procedures in the least.

Going back to seven paragraphs (of disinformation), I wrote:

The LNDD had at least three methods available to make the identification of the peaks in the IRMS compliant with the TD2003IDCR, and chose to do none of them:

1. It could have used a cal mix in the IRMS that had all the analytes of interest, and matched RT and/or RRTs on the same instrument. The LNDD chose not to do this.
2. It could have used nearly identical chromatographic conditions between the two instruments to make comparisons easier. The LNDD chose not to do this.
3. It could have used cal mixes on each instrument with a known peak after the last peak of interest, and computed Kovat's indexes for all the peaks in between the anchors at each end. The LNDD chose not to do this.

The LNDD did none of these things to put itself in compliance with TD2003IDCR, and the panel concluded therefore, that TD2003IDCR does not apply. This is like saying a motorist who has an inoperative speedometer cannot be given a citation for speeding because he didn't know how fast he was going. It is his responsibility to know. It was the LNDD's responsibility to produce an identification that met TD2003IDCR, and it did not do so by any reading of the standard.

The reasons why conformant identifications were not provided are excuses for not doing objective laboratory work. This is a "no liability" reading of the ISL compared to the strict liability of the Code as applied to athletes. If an athlete doesn't know what is in a supplement and tests positive for a banned substance, no excuse is good enough. If a lab doesn't know how to meet the ISL for peak identification, any excuse is accepted, and the athlete is still "deemed" to be guilty.

TBV

Larry said...

BLT -

I thought your last post was a terrific post, even if it WAS a bit random! (jk)

You raised the question of our purpose here. My purpose is to engage M in a legal discussion of the issues he raised, and maybe a few other legal issues as well. I am an attorney with no experience in sports law and an obsession with the FL case that is borderline unhealthy. I have come to the internal conclusion that the FL arbitration decision is incorrect as a matter of law, and I'm shamelessly using the hard work posted here by "M" to see if I can prove my conclusion to the satisfaction of anyone else.

Of course, a lot of folks here have reached the same conclusion I have, based on similar reasoning (Mike S., for example). But one of my goals here is to be able to state why the FL decision is wrong, in the simplest possible terms and in the shortest amount of time -- while standing on one foot, as Rant would say. This is my answer to your question on how to present information at the CAS.

BLT, the example I used about the dirty test tubes was just an illustration to try and show how one might reasonably try to overcome the presumption that an ISL violation caused a doping finding. I don't think that LNDD uses dirty test tubes! Your larger point here is a good one: we can't know everything that took place at LNDD that might have impacted the FL case. Moreover, from a legal standpoint, it would be a waste of time and money to approach every doping case with the attitude of "leave no stone unturned." (IOW, I'm willing to assume that LNDD's water supply is pure.)

DBrower said...

BLT,

I was kind of hoping discussion about what should be argued in the appeal would happen in our appeal brief, but that hasn't really started.

I'll be happy to do another post of collected thoughts of the community here if that might help.

TBV

Larry said...

TBV -

LMAO about the Mr. Idiot story! Damn. That's FL all over. Mike, nice to know that one of my favorite posters on DP is also one of my favorites over here.

If "M" is patient enough to continue this dialog (and seriously, the guy deserves a prize for what he's doing here, sort of like jumping into a lion's den with a 3 inch steak and a Powerpoint advocating a vegetarian lifestyle), we'll get into a discussion of the legal meaning of TD2003IDCR. I'm trying to lay the groundwork for that discussion.

While the FL legal team are better lawyers than I've ever thought of being, in hindsight I think they should have asked the question: "what criteria did LNDD establish as required under TD2003IDCR to identify IRMS compounds?" Maybe that should have been the first question they asked to all of the USADA witnesses. Because it looks like LNDD failed to establish any criteria, or LNDD's criteria were not acceptable under the TD. Then they could have skipped about 6 days of the arbitration, and we could spend all of our time discussing biological passports or something (cycling is God's way of making sure that we have an endless source of blogging topics.)

By the way, the chain of custody issue ALSO turns on the criteria of procedures that LNDD is required to maintain under ISL 5.2.2.2. For some reason, no one involved in the FL case seemed to be interested in whether these procedures existed or what they had to say. Instead, all focused on the WADA chain of custody guidelines set forth in TD2003LCOC. These guidelines set up parameters that the labs are supposed to follow in setting up their required chain of custody procedures. The guidelines are not intended to be the procedures, and as we saw in the FL decision, they don't work well as procedures.

Thank you everyone for giving me the background information I requested.

Larry said...
This comment has been removed by the author.
m said...

DailyBob,

Let me try asking you again to actually look at the chromatographs for F3 and tell me why the peaks don't match. The RRTs are off by about 4%, but it is the same sample in both. They have to have the same metabolites in both. There is no secret ingredient. What could those 4 central peaks be other than the 5B and 5A? Really nobody has been willing to address that issue and explain why OMJ and RH are wrong. Maybe, because they don't have any answer that they like. Even one of the science guys on DP who claims Landis is innocent says the identification is reasonable ( I can't remember his name but he posted here also.)

The closest I've heard is the remote possibility that the peaks were transposed because the ramp temps were slightly different. But those who are familiar with these metabolites really discount that possibility given the small difference in temps.

Larry said...

M -

With all due respect, the "what else could they be" method of identifying compounds does not satisfy the requirements of TD2003IDCR.

But I'm getting ahead of myself. You and I are still working on an analysis of your Section 3A.

the Dragon said...

M,

If doing visual matches is the best recognition method, please explain why does a Technical Document even exist? Seems a quite worthless exercise.

Regards,

the Dragon

DBrower said...

Folks, it's getting a little snippy here, please be more thoughtful. I'd like to call your attention to some details present in An emailer looks at chromatograms. There are some good insights there.

TBV

m said...

Larry,

My question to Dailybob is not about the law or a technical standard, but about whether science tells us that Landis doped. OMJ didn't really care about the technical standard or the law, he was concerned with whether the scientific results showed a proper identification of the metabolites. He said yes based on the visual pattern of the peaks.

So I'm asking for a rebuttle wrt to the visual evidence.

Mike Solberg said...

m, from your 3:05 post, I think the guy you're talking about is Kevin/Duckstrap. If I remember right, he says that it is likely (although not certain) that those peaks do contain some 5A and some 5B, but there is no way to know whether they contain anything else, and there IS other stuff in there they could contain, and various ways the ion values could, incorrectly, come back so negative. The way to rule out such contamination is to show the complete mass spectrometer data, which (surprise!), LNDD has never produced and which probably no longer exists if it ever did.

To my way of thinking, if the question you are ultimately asking is "Did Floyd dope?" then there is only one possible answer: We don't know. Only Floyd and God know. End of discussion.

Given that, the question we can reasonably discuss is "Given the available evidence, should the case have been decided the way it was?" Or, "Does the evidence prove, to the necessary degree of proof, that Floyd doped?" I don't think it does.

syi

Larry said...

M -

Understood. If my last post was snippy, I apologize. You're doing great work here, and I appreciate being able to discuss these matters with you.

From one lawyer to the other, I think that the "science" of mass spectroscopy is, from a practical standpoint, less than perfect. There's some art to this as well as science. You need to do a bunch of preparation to the sample, to filter out the things you don't want to measure. You have to do the injection just right. Then there's all the discussion of "good chromatography" and "bad chromatography", peaks with "shoulders" and all that. Not to mention the manual data adjustments to remove background noise, high sloping baselines, matrix interferences, time gaps between reference tests and sample tests, and a myriad of other factors. This isn't like taking a fingerprint!

If you go back and look at the testimony of the lab technicians, you'll see repeated instances where they tried to do X, but forgot to do Y, and had to start all over again. I think that a lot of people concluded that these lab techs were incompetent, but I think a more accurate conclusion is that they were performing a difficult and complex job under time pressure. There's just a lot that can go wrong!

I'll quote from the testimony of Ms. Frelot:

"In order to prepare the IRMS in the morning we fill a LN and then we verify the peak center, and when it is good, we launch the stabilization. I had forgotten to verify the peak center, when I noticed that, I verified the peak center and then to verify the center you have to open up the 22 CO2 valve, in order to do the peak center, and then you have to close the valve and I had forgotten to do that on the second one, which means I then closed the valve and then performed the stabilization."

So, what would have happened if Ms. Frelot had never opened the CO2 valve, or forgot to close it, or did not close it at the right time, or closed it part of the way? I don't know. Presumably it would have affected the results. Maybe it would have affected the results in an obvious way that anyone could see, or maybe it would have affected the results in a subtle way that's hard to detect. In the latter case, we'd have received a graph from LNDD with peaks neatly labeled, with numbers showing accuracy to two decimal places, and maybe everything would appear to line up visually. But the numbers and the peaks would be no better than the test procedure that created them.

I, too, was hoping that during or after the FL case, we'd hear a loud and clear consensus from the scientific community that the science used in the majority opinion was right, or that it was wrong. I have not heard anything like this. There are people like OMJ, who make a persuasive case in favor of the majority, and people like the "emailer" that TbV mentions above, who make a persuasive case in the other direction. As lawyers, you and I are used to seeing cases like FL turn on a "battle of the experts." But this battle is crazy! We don't seem to have agreement on the most basic kinds of questions, such as whether a RRT analysis is valid or invalid. What is going on here?

One possible explanation for the lack of consensus is that the scientific experts are approaching these questions from very different points of view. If a scientist uses mass spectroscopy for research purposes, or even for medical diagnostic purposes, then it's probably OK to run these tests with a more relaxed methodology. The researchers are probably using the tests to study a representative number of samples, so an error on the analysis of one or two samples may not matter much. Similarly, in medicine, a doctor will use mass spectrometry along with a host of other diagnostic tools, to avoid the consequences of a false positive or false negative. So even the doctors may have some tolerance for error. And perhaps for the doctors and the researchers, it's good enough to identify peaks based on visual patterns.

In contrast, the scientists who do forensic work seem to take an entirely different attitude. These are the guys who insist on strict process, detailed chains of custody, beautiful chromatography and the like. Why? Because they can't afford to be wrong. Too much is at stake: a person's livelihood, or liberty. Also, these scientists know that their results may face more scrutiny than "peer review" -- they're going to have to stand up to cross examination and judicial review. So, I think these scientists are not necessarily going to see eye-to-eye with the scientists who use mass spectrometry for research purposes.

You ask the fair and legitimate non-legal question, what else could the identified IRMS peaks be, if they're not 5B and 5A. The answer is, who knows? Take a look at the relative retention times II IRMS graph cited above by tbv. You can see 4 clearly defined peaks in the reference graph, and something like 18 peaks in the FL sample. If 4 of those 18 peaks are the ions we're trying to measure, then what are the other 14 peaks? I've asked this question, and the best answer I've received is that they're "something else". So, when you ask the "what else could those 4 peaks be" question, I'll give you the same answer I get when I ask about the other 14 peaks: they could be something else.

So, to be honest here, I don't trust the scientists to tell us whether FL is guilty or innocent. If they can't see eye to eye, if they can't tell us whether we can or cannot use RRTs, if their tests are so complicated to run and so easy to screw up, what do we really think these people can tell us?

Your question was not a legal question, but from my vantage point, the best answer to your question is a legal answer. At some point a bunch of scientists sat down and wrote the ISL, including TD2003IDCR. They did not have the facts of the FL case in front of them, they were just trying to figure out how to best describe when we can trust the mass spectrometry test results and when we can't. The cycling world agreed to live with these ISLs. The ISLs are not perfect, heaven knows, but in the absence of some unified statement from the scientific community, they're all I have to go on. And the ISLs say, if you're looking at an IRMS graph and you can't identify a compound using criteria satisfying the requirements of TD2003IDCR, then don't trust the graph.

So ... I don't trust the graph.

This is probably not the "rebuttal" you're looking for, but it's the one I got!

DBrower said...

Along with Mr. Idiot, I think if we start looking at the question of actual innocence or guilt, then we have lots of other things to discuss -- what would be corroborative evidence, to use a term, but which may not be relevant in the legal inquiry.

For example, the mass-spec data that was destroyed may have been exculpatory in showing the presence of contaminants. We note that mass spec data from the other tested B samples has also not been provided. The rationale given is that it is not required by the ISL, but if we are interested in the truth, why isn't it offered?

Similarly, the raw data files were never made available, and the rationale was that Landis might alter them -- as if that were possible with data on a CD-ROM. Not that it is of much use, as the data on the CD that was taken has already been massaged by Mongongu and Frelat. However, neither have we seen the comparable data from the other B samples, which someone interested in truth would certainly keep, and make available.

Landis has still not gotten the SOP of the test that would identify the details of the chemistry. Considering the theory the separation chemistry isn't good enough is hard to evaluate without the details.

These are pieces of hard, raw evidence that certainly did exist. They are potentially exculpatory in revealing what really happened. Yet some that did exist have been destroyed, and others just withheld.

Julich at DPF splits the middle on this. He'd like the Landis camp to concede the identification issue, which leaves them little to work with in the legal, ISL violation rule system. On the other hand, he doesn't really consider the non-admissible problematic evidence, because it isn't admissable. I think this reflects his basic predisposition that the lab basically did the right thing and that Landis doped, not an evaluation of the totality of the available (and unavailable) data.

If we want to consider true culpability, I'd want to look at the game playing in the litigation, which appeared (from my admittedly biased perception) to be mostly stonewalling on the USADA side, including destruction of evidence, refusal to provide potentially exclupatory evidence, novelties like the irrelevant B sample tests, arguing against points made by Landis since the ADRB filing that were conceded in testimony by the LNDD witnesses, and arguing against the deposition of the LNDD witnesses before the hearing in a way that cost Landis much time. Then there is the mud slinging of the Papp and Lemond testimony which had no relevance to the scientific truth -- a tar pit which drew Geoghegan in like a black hole.

All of the USADA behaviour smells like a coverup to me.

On the other hand, Landis has been open about the case and evidence in an unprecedented way, and has generally behaved like an artless naif who was innocent and thinks this all came about as the result of a lab error that he can't get the data to unequivocally prove.

If I'm in a legal proceding, I'm not going to stipulate anything about the identification, because it is the main lever to pry open the ISL violation box that leads to a burden flip. A truth-seeking exercise would then consider all the other data available as part of preponderance of evidence and comfortable satisfaction. I submit such an effort ought to result in a very queasy feeling about the findings.

I will also concede that the case is unlikely to become a truth-seeking exercise, because the WADA rulebook doesn't want that to happen. It wants to convict those accused of doping, and procedurally stacks the deck against the athlete.

It is particularly telling that there is no "general sense" provisions that allow challenge on things like whether the protocol is, in fact, valid; whether the technique is "fit for purpose". We learn that no-one reviews the test protocols outside the laboratory, and there is no accountability for their correctness, as long as the lab executes them to the limited set of things defined by the ISL.

Now we are told, in the case of identification, that even things for which there appear to be rules can be ignored when it is inconvenient to obtaining a conviction. LNDD could have cleanly identified the peaks, but didn't. So they don't have to; there doesn't appear to be a requirement to prove the peaks are pure, so Landis can't even argue the issue.

This is not a fair game, it is slanted house rules in action.

Given a solid issue in identification, it is unfair to expect concession on it unless all the other issues relevant to truth are allowed back on the table.

TBV

Unknown said...

M/Larry,

Sorry it took me so long to get back. M calling me "Daily Bob" is completely appropriate, because with my insane life, I'm only going to be able to weigh in once a day at best.

M, You ask "What could those 4 central peaks be other than the 5B and 5A?" Those central 4 peaks probably contain 5A and 5B. The problem is that the different temperature ramps have increased the probability that 1.) the peaks have coeluted with other things in the urine and 2.) the peaks may have changed order. This is why, it's so necessary to anchor the peaks to a known reference standard, because the only way to have confidence that the peaks are what you're looking for is to have them elute within a very narrow time window when compared to either 1.)the known metabolite in the cal-mix and/or 2.) the time difference between the reference standard and the metabolite in the sample. If you do this, and then then combine these results with the mass spectra (which I understand were not supplied for some reason), your confidence can be high that you have a positive ID. I know this seems very "black or white," but it's the whole basis upon which the machines are used to identify materials

In regard to OMJ and RH, I haven't spent much time over at DP, but from the few things I've read over there, they both impress me as very knowledgeable (as does Duckstrap). I don't know why they're comfortable doing this visually. I will show the chromatograms to one of my Analytical Chem buddies at work to make sure I haven't totally missed the boat someplace. The soonest I can get back will be tomorrow night or Saturday sometime.

Larry, It's late here, so I can't address all of your questions now, but the answer to your first paragraph question from your 1:05 post is: you have it correctly.

Best

the Dragon said...

Larry,

I think I was actually the snippy one.

Dang...If I had waited for your post, I could have avoided the last 2 hrs of re-reading Dr. Brenna's testimony(not done yet).

It's interesting reading again after the conversation between you and "M". So far I have only found one direct reference to RT which came in answer to Mr. Youngs question. On page (25 line page#) 256 Lines 18-22, where Dr. Brenna specifically says RT is how you place the peaks. Yet in reading the rest of his direct testimony, since the "visual" question was never specifically addressed, I can't rule out he might be relying on the "visual" method.

If so, it's interesting under cross by Mr. Suh, he didn't notice that a number of the test substances were out of accepted range, further verification of the use of "visual" method?

Here I've spent my whole life in awe of science and it's precision (they did name a Method after it). Maybe my mundane debits & credits ARE just as precise, at least they always balance :-)

Regards,

David

DBrower said...

You need to do a bunch of preparation to the sample, to filter out the things you don't want to measure. You have to do the injection just right. Then there's all the discussion of "good chromatography" and "bad chromatography", peaks with "shoulders" and all that.

When I asked around, I learned that "bad chromatography" is essentially synonymous with "bad chemical separation in the preparation". The dirty baselines and sloppy shoulders are symptoms of inadequate chemistry, not some missing magic touch on a knob on the machine. When the say manual processing is no substitute for good chromotography, what they mean is you can't clean up dirty fractions after the fact in software or with manual adjustment of the parameters. It is just a badly prepared sample being injected, producing noise and unreliable results.

For those wanting to look at the visual identification standard, I repeat my pointer to an emailer looks at the chormatograms. He says, "the graphs displayed in the majority decision look odd and inconsistent." and thinks, by visual inspection, that they suggest at various times

1. Bad chemistry.
2. Someone else's sample.
3. That relative peak heights aren't valid for pattern matching.
4. That peaks may swapped between the machines.
5. Peaks in the GCMS that have vanished in the IRMS and vice-versa, leaving little idea whether each peak is pure, or exactly what is in each peak.

TBV

DBrower said...

in hindsight I think they should have asked the question: "what criteria did LNDD establish as required under TD2003IDCR to identify IRMS compounds?"

I think Suh tried, but I agree is wasn't clear enough. I felt at the time he was baiting a trap that he'd close later. I think he botched his cross of Brenna's rebuttal, probably because he was pressed for time. As with much of the case, I think the information is there, but it isn't as clear as it needed to be, and this gave the wriggle room that, from my point of view, allowed the Majority to attempt the arguments they made.

The first Suh mention of identification with Brenna is at the transcript in PDF page 149, transcript page 255, line 11:


11 Q. And what are the three peaks of
12 interest there?
13 A. Same three.
14 Q. And how would I know --
15 A. 5-alpha, 5-beta --
16 Q. And how would I know which is which,
17 because they just have numbers at the top.
Page 149
MAY 14-23 2007
18 A. Well, they have retention times that
19 match on the previous -- with the previous
20 GC/MS, and the GC/MS delivers structural
21 information, like aliquots and so forth, that
22 tell us which is which.
23 Q. So by the numbers at the top, can
24 you identify those for us? Like which is the
25 5-beta?
RESLING SPORTSCRIPT (719) 632-6391

256
1 A. This one, 5-beta, 5-alpha, pdiol.
2 Is that what you're asking me?
3 Q. Yes.
4 A. Good.


And that was it. Suh didn't make a big deal of it; he appeared to get the answer he wanted, no more, and let it sit in the transcript, where he pulled it out to impeach the visual identification standard during the rebuttal cross. I think he did a duct-tape job of making the point, though, and Brenna said enough the Majority could peel him off the wall without adknowledging the contradictions.

TBV

m said...

Larry,

"You can see 4 clearly defined peaks in the reference graph, and something like 18 peaks in the FL sample. If 4 of those 18 peaks are the ions we're trying to measure, then what are the other 14 peaks?"

Larry you don't understand the science.

The four central peaks elute around 1300+ seconds. 4% error on the RRT means that those central peaks are off by about 50 seconds. There are no other peaks within that range.

They used a chromatographic standard 5A Androstanol and then measured the RT of the 5A and 5B metabolites from that reference. All those 18 peaks you are talking about occur way before the 5A and 5B, or way after. There is no way they could be the 5A and 5B unless some scientist has an explanation. Which is why I keep asking.

m said...

Dailybob,

"increased the probability that 1.) the peaks have coeluted with other things in the urine and 2.) the peaks may have changed order."

Based on OMJ and RH, I believe that the probability of your 2) is minimal, certainly not 51%. How would you determine the probability of this? You know the temperature ramps, you know metabolites. You could run it very easily and see if the peaks switched.

As to your 1), OMJ concede that was a possibility but he would want to know what special substance was in Landis' urine that it would always coelute at the same RT. Remember there were positives for multiple stages. He thought it was very unlikely such a substance existed, and it was up to Landis to come up with some plausible substance. We know from the blank urine sample that there was no normal urine substance that eluted at those RTs.

wschart said...

I may be missing something but it still seems to me that the issue of whether retention times or relative retention times can be used to ID the peaks in the Landis tests is being used or not used for the convenience of supporting whatever side of the case a person is on. Brenna talked about matching RT until M-A seemed to shoot that down, then switched to the visual match idea (which by the way seems to me to just be a fuzzy sort of RRT). M and others are saying that eyeballing the graphs shows a match, but in his 11:12 post talks about RTs too.

Larry's discussion of differing standards used by research/medical tester vs forensic testers makes a lot of sense to me and does explain why we see no consensus here.

The question now becomes one of what standard is required by the WADA game. While we might like that the standard be the same or at least similar to the forensic standard that would apply to a criminal case in the US court system, that might not be the standard that applies here. You have to play by house rules, like 'em or not. It also seems to me that the WADA rules are vague enough that it is difficult to reach a consensus.

m said...

Larry,

Rereading my last comment to you, I may have sounded a little snippy myself. Not my intention.

WSchart,

I wonder if the forensic standard is much different. You should see the stuff that gets admitted in trials. Also, their key witness Meier was an academic researcher, as was Brenna. Goldberger, is a practiced expert witness and knows how to sound persuasive. I wonder if his lab is really run any better. You just have to read of some of the scandals that have occurred in various police labs across the country to know that they are not foolproof either.

Larry said...

M -

LOL! No offense taken. To say that I don't understand the science is about the most accurate statement anyone ever posted on a blog. I think that for the moment at least, I'll let tbv, Mr. Idiot and other handle the science, and I'll just be the lawyer.

Luckily, as a lawyer, I don't have to understand the science. I have to understand what the rules tell me about the science, and what the witnesses tell me about the science.

The rules tell me that if you cannot identify the peaks using a method that satisfies TD2003IDCR, then you have a bad IRMS chart that you can toss in the waste paper basket. As a lawyer, that's all I need to know. As a curious human being, I'll wonder what the peaks mean and if anyone can explain them, I'll enjoy the conversations we're having here and I'll even participate in them. But as a lawyer, so long as the chart was not prepared in compliance with TD2003IDCR, then the chart is worthless.

As a lawyer, I'll listen to the witnesses too. OK, the rules tell me that I should ignore the IRMS chart, but maybe the rules are wrong, or they don't apply to these facts. So, if the witnesses tell me that the rules are wrong and that I can do a visual identification of peaks, then as a lawyer I might have to rethink my position. However, the witnesses here are all over the map. Some are like OMJ, some are like tbv. As a lawyer, I cannot reach any firm conclusions from listening to the witnesses.

Now, you might tell me that as a lawyer, it's my job to evaluate the testimony of the witnesses and reach a conclusion as to which witnesses are credible and which are not. I know that sometimes, my job as a lawyer requires me to do that. However, I don't need to do that here. WADA's own TD2003IDCR sets forth standards for how compounds are supposed to be identified. As a lawyer, my evaluation of the evidence is that there is no clear scientific consensus that TD2003IDCR is wrong and that a different procedure is right. So as a lawyer, I'll stick with the rule under the TD.

I know that you were asking to discuss the science and not the law. But I don't think the science and the law are that easy to separate. TD2003IDCR is law, written by the WADA scientists from a relatively objective science point of view. They weren't thinking about the FL case when they wrote this TD. They were just trying to come up with rules that their labs could live with, that would do a reasonable job of identifying the dopers. And the odd thing is, with all of the controversy surrounding this case and the WADA rules, I've seen nobody criticize this TD. To my knowledge, OMJ is OK with this TD, and so is tbv.

Why not stick with the TD as a reasonably decent science rule, and move from there?

m said...

Larry,

Our discussion involves both law and scientific evidence, and possibly policy questions as those might inform our interpretation of the law and evidence. I'm quite prepared to discuss TD2003IDCR.

My question to Dailbob was a pure science question.

Larry said...

Question to all: the arbitration transcript refers to the LNDD SOP (standard operating procedure). I know that this would be in French and impossible for me to understand, but is this available anywhere? I can't seem to find it.

Also, there are a number of exhibits referenced in the transcript that I can't find anywhere, including an Exhibit 112, that I'd love to see and cannot find. Anyone know if these documents are available?

Mike Solberg said...

m, have you been able to look carefully at the "An emailer looks at the chromatograms" post yet? If so, can you still trust the visual matching method? That post, based in LNDD's own work on Floyd's samples, seems to clearly show that peak height can and does change between GCMS and GCIRMS, and that position/order may change also. How can a visual match still be reliable?

m said...

Mike Solberg,.

BTW it was Duckstrap I was thinking about. Thanks.

I skimmed them at the time, and the short answer is no. Unless you can point me to a particular graph.

The differing peak heights didn't bother me. I believe Dailbob said the IRMS heights might not be the same as the MS since IRMS measures combusted carbon atoms. I really don't know enough to evaluate that claim one way or the other.

His claim of switched peaks didn't make sense to me. There was no switching, just a stretching out. He pointed to two doublets which became "4 evenly spaced" peaks. First I might not have characterized them that way, but RH on DP said one might use slightly different ramp conditions to get better separation of the peaks. Stretching out of the 2 doublets to 4 peaks in the IRMS is consistent with that idea. In fact I think that may explain why the retention times are a bit longer in the IRMS. There is no switching of peaks that I can see. Moreover, the peaks he pointed to were not even any of the analytes.

His claims that the samples might have come from different persons? Well he wasn't even an organic chemist. I don't want to mention green helicopters, but that was my first reaction.

But I haven't had time to parse them all carefully, and of course I am no scientist.

wschart said...

Considering the problems LNDD had with sample numbers, it isn't out of the question that there could be 2 different persons. This issue wasn't raised, as best I recall, by the defense at the hearing, but it has been pointed out here and elsewhere. At least one of the sample numbers in question has been alleged to match another rider. BTW, I think you mean black helicopters!

Unknown said...

M,
I went over and looked at the chromatograms with my friend in our Analytical Chemistry department, just to make sure I hadn't totally missed the boat on something. First, you'll be pleased to know that he clearly saw your point that, because those four peaks were by themselves, they most probably were the same central four peaks as on the GC/MS. However, like me, without the metabolites in the cal-mix, he feels you can't prove it. Also, without revealing my position, when I asked him if he would vote to convict Landis with this, he said no. This must seem incredibly anal to you, but it has to do with the the principals upon which GCs work. Because there's always slight variability, even when running the same sample on the same machine with the same column, it's critical (i Can't emphasize this enough)to be able to compare the elution time in the sample to that in some standard (the cal-mix in this case) contemporaneously (sp?). Without it, you just don't have definitive proof. You're left with just being able to speculate.

Larry asked me (in his 1:05 post yesterday) whether I thought there was an IS violation. While this is a question better answered by a lawyer, from my perspective as a scientist, the requirement of running the same substance in a spiked urine sample was clearly not met if all three ions are required to be identified

M, you asked me whether I thought the science tells us whether Landid doped. I know this seems like a cop out, but I honestly don't know (or feel that I can tell with any surety). I think this is due to my view of this test not being done in a way that is definitive combined with the fact that I was influenced by Amory's testimony. Together, there's enough doubt in my mind, that I certainly wouldn't vote to covict.

Hope this was helpful.

m said...

Dailbob,

Thanks.

That the RTs were off doesn't bother me that much. That can be explained by the differing GC conditions. The 4 central peaks match, there is no other substance eluting nearby that anybody is claiming could be a match. So we can't really claim that they've misidentified the metabolites. In addition, we have the same results for 3 or 4 other samples. So my subjective probability on the identification issue is 95%.

Any possibilities regarding co-eluting substances or switched peaks is not a function of the RTs being off, but of the missing mass spectra and slightly differing GC conditions. But really the probabilities of these occurring or having affected the results are small according to OMJ.

As to Amory, I found his testimony at best suggestive and anecdotal, and there were counter anecdotes showing pdial values just like Landis. But I can understand how it might have raised some doubts in your mind.

Again thanks for your perspective.

Larry said...

M -

So you'd like to do the legal analysis involving IRMS identification? Good! This is really very simple ...

The simple explanation

1. The ISL includes TD2003IDCR. See ISL paragraph 1.0.

2. TD2003IDCR requires each WADA lab to "establish criteria for identification of a compound." Once established, the criteria becomes part of the ISL for the lab.

3. I can find nothing in the transcripts, briefs or other publicly available material to indicate that LNDD ever established such criteria. The failure to establish such criteria is a departure from the ISL.

Alternatively, the criteria established by LNDD for identification of compounds in IRMS testing failed to satisfy TD2003IDCR, and that is also a departure from the ISL.

4. Under WADA rule 3.2.1, USADA was required to prove that this departure from the ISL did not cause the adverse finding.

5. USADA failed to meet the burden of proof referred to in (4) above.

Case dismissed.

The longer explanation

My guess, M, is that you're not going to be satisfied with my simple explanation. You're going to want to see something more complicated. TBH, I think that the simple explanation works just fine, and that any more complicated explanation requires me to make a better case for USADA than they made themselves. But I'll give it my best shot.

(1), (2) and (4) above are just restatements of the rules, which we've already discussed. (3) and (5) are relatively obvious findings of fact that emerge from a reading of the trial transcript. It will take two posts to fully describe these facts. We'll start with point (3) above, to show the ISL departure based on LNDD's failure to establish and follow a satisfactory set of criteria for identifying compounds. I'll turn to the burden of proof issue under (5) in a later post.

LNDD had no criteria for identifying IRMS compounds

Did LNDD have established criteria for identification of compounds as required by TD2003IDCR? I only wish that someone had asked that question directly during the trial. But there's a great deal of indirect evidence that LNDD had no such established criteria. If such criteria existed, they should have been included in LNDD's standard operating procedure (SOP). The SOP does not appear to be available to us (it would be in French anyway), but the two LNDD lab technicians (Frelat and Mongongu) both testified in detail regarding the steps they took under the SOP when they performed the IRMS analysis. How did they identify the peaks shown on the IRMS graphs for the FL samples? I can find not one word of testimony from the lab technicians on this point, not in hundreds of pages of transcript.

Let's look at the testimony of Cynthia Mongongu, beginning at page 390 of the transcript (271 of the pdf file). Ms. Mongongu was the lab technician who performed the IRMS analysis on the Stage 17 A sample (Esther Cerpolini did the GC/MS), and verified the analysis of the B sample. Her testimony was longer and more detailed than the testimony of Ms. Frelat, plus Ms. Mongongu was the more experienced and knowledgeable lab technician, so I've focused more on her testimony. This testimony is lengthy (about 105 pages) and focused almost exclusively on how she performed the IRMS analysis. From her testimony, she took the following steps:

a. She prepared the sample, following the LNDD SOP.
b. She performed the GC/MS analysis for the identification of the compounds. According to her testimony, she verified that the GC/MS instrument is operating properly, by first making certain that the MassLynx is functioning properly and the ion source is functioning properly, then she checked the instrument for leaks, and she injected a mixed acetate into the GC/MS to make certain it was operating properly. Once the instrument was verified, she injected the prepared FL A sample, in fractions, in a prescribed sequence along with fractions of blank urine samples.
c. She performed an IRMS analysis. This involves verifying that the instrument is set up to verify the CO2, and that the instrument is stable. Then the instrument is calibrated with reference to a test on the mixed cal acetate. Finally, FL's sample is injected into the IRMS.
d. She performed a data reduction process on the data, consisting of verifying the background noise, and verifying that the chromatograph peaks are correctly integrated. This is the point at which Ms. Mongongu would make manual adjustments to the data.
e. At this point the results of the test are calculated, and it was determined that FL's A sample was positive for exogenous testosterone.

Please tell me, where does Ms. Mongongu describe how she identified the IRMS peaks? She does not do so.

On cross-examination, Maurice Suh walked through all of these steps again with Ms. Mongongu. (see testimony beginning on page 556 of the transcript, page 421 of the pdf file) If, somehow, the work Ms. Mongongu had performed to identify the IRMS peaks was omitted in direct examination, Ms. Mongongu had the opportunity to provide this information during cross-examination. In re-examining the steps taken by Ms. Mongongu, Mr. Suh was careful to walk through these steps in sequence, making certain that no step was skipped. So, Mr. Suh started with the stability runs on the IRMS machine, and then asked Ms. Mongongu to describe each step she'd performed, in sequence, using questions structured like "After you did x, then you did y, correct?" Again, there's not a word regarding how Ms. Mongongu identified the peaks in the IRMS tests.

You can also examine the testimony of Ms. Frelat, which is shorter but substantially the same as the testimony of Ms. Mongongu.

It's worth noting that Dr. Meier-Augenstein also wondered out loud how LNDD went about identifying its IRMS peaks. On direct examination (transcript pages 1418-19), Suh asked Dr. M-A: "When you look at the document package, do you see any documents which help you identify how they decided which one of these peaks constitutes the internal standard?" Dr M-A responded "No."

The cross-examination of Dr. M-A is even more damning to USADA's position, which is surprising given that USADA appeared to INTENTIONALLY use this cross-examination to prove that LNDD had no criteria for identifying compounds. When Dr. M-A tried to argue on cross-examination that LNDD must have had SOME procedure for the identification of the IRMS internal standard peak, USADA's counsel seemed intent on proving the contrary (testimony p. 1517, pdf 1291):

"Q. And where is it in the Paris laboratory's specs that says when you use the internal standard in connection with identifying retention time in a blank urine or athlete sample that it has to be within a particular delta value spec?"

Dr. M-A eventually responded as follows (transcript p. 1518, pdf page 1292):

"A. I -- I don't know. I don't know how they do it. Because, as I would say, there's nothing in the specs, so I don't know. Is it divine intervention or they just pick one? How do they -- how do they know which peak is their internal standard? They must have -- they must have some -- some criteria to say, Oh, let's see, there's one peak at this time; there's one peak at this time; that's not the peak at this time. They're also 20 seconds apart. How do they choose which of those it is? I don't know. And you're quite correct. There's nothing in the specs that actually tell you how to do it."

At this point, USADA's counsel could have easily rebutted Dr. M-A and shown him that LNDD DID indeed have criteria for identifying the internal standard. USADA did not do so. They appeared intent on making the point that no such criteria existed at LNDD.

I don't see any other testimony relating to whether LNDD has the established criteria required by TD2003IDCR. The testimony of other witnesses (like Dr. Brenna) did not speak directly to what LNDD did or what procedures LNDD had in place to identify compounds. Instead, this testimony spoke to what are acceptable identifying techniques as a general matter. (This other testimony will become relevant in a later post, when we look at whether USADA effectively met its burden of proof under Rule 3.2.1.) You can read this other testimony, and perhaps you can point out to me whether or where these other witnesses spoke to what LNDD actually did. In any event ... the testimony of Ms. Mongongu and Ms. Frelat is the best evidence of LNDD's actual practices and policies, though the testimony of Dr. M-A cited above IS instructive.

I think my conclusion here is a fair one: if Ms. Mongongu or Ms. Frelat took ANY steps to identify the IRMS peaks, these steps were casual and not systematic. There's no evidence that Ms. Mongongu and Ms. Frelat followed anything like "identification criteria" to identify these peaks. I conclude from this that in all likelihood, LNDD had no such identification criteria. LNDD's failure to adopt identifying criteria under TD2003IDCR is a clear and obvious departure from the ISL.

If LNDD had identifying criteria, the criteria failed to meet TD2003IDCR standards

But ... let's assume I'm wrong. Let's assume that LNDD DID establish TD2003IDCR identification criteria. What might that criteria be? Before we get started on this part of the analysis, I'll issue a warning: even if we assume that the LNDD had established TD2003IDCR criteria, the evidence of what this criteria might be is skeletal and not entirely consistent. I STILL think the best evidence is that no such criteria existed. However, if there WAS such criteria ... the best evidence I can muster is that any such criteria required a relative retention time analysis.

In her testimony, Ms. Mongongu testified on two occasions that LNDD computed included an internal standard in its testing in order to calculate relative retention times. (testimony p. 434, pdf p. 311; testimony p. 653, pdf p. 509). Here is one quote from her testimony:

" Q. Why do you add an internal standard to the blank urine and the athlete's sample?
A. The internal standard allows us to calculate the relative retention time of the molecules analyzed.
Q. Is that to make sure that you're looking at the right peaks?
A. Absolutely, yes."

You can also look at the IRMS SOP described above. The second step under that SOP is to perform a GC/MS analysis. What would be the point of doing such an analysis? The science types can weigh in here, but in the context of an IRMS test, a GC/MS analysis would be performed to provide peaks that could be used to identify the compounds being measured in the IRMS test. I don't know why else a GC/MS test would be relevant, other than to provide retention times that could be compared in some manner to the retention times for the IRMS test.

Dr. Ayotte's testimony provides corroborating evidence. On direct examination, Dr. Ayotte testified that she understood from Ms. Mongongu's testimony that LNDD ran a 5aA internal standard test, and that this was a good practice "to determine the relative retention time of your analytes." She went on to testify that this testing is "a necessity, otherwise, you don't know what you are measuring." (testimony pages 811-12, pdf pages 652-53) Dr. Ayotte testified more than once regarding the link between the internal standard test and the determination of relative retention times (see, for example, testimony page 849, pdf page 686).

So, if LNDD DID have established TD2003IDCR identification criteria, this criteria was based on relative retention times. The next question is, what procedures were established under these criteria for comparison of relative retention times? On this question, we're stymied. As I've already stated, there's not a word of testimony on what LNDD did to identify the IRMS peaks. However, the assumption (in the majority opinion and on this forum) is that LNDD identified the IRMS peaks by comparing them visually to the peaks identified in the GC/MS test. The best explanation I can find for this technique was provided by Dr. Brenna in his cross-examination (transcript page 1971, pdf page 1702):

Brenna: You can't use relative retention times. I explained why this morning. Would you like to know what to use? I can explain it. We have the chromatograms up there.
Suh: No, that's all right.
Brenna: Okay.
Suh: Actually, no, why don't you go ahead and explain it. Go ahead.
Brenna: Well, on the GC/MS side, we see a pattern, so we can see peak heights. And so -- and we want to look at the overall pattern is what -- an intermediate-sized peak; a small peak; this is one of the strong peaks; and then a large one. And then we move over a bit, and we find a large peak, an intermediate peak, a smaller peak. And then we move to the end, and we see a large peak. And if you look, you see the same pattern here. In fact, I would -- I would say, you see a number of peaks here, a number of peaks here, a number of peaks here, and they have approximately the same -- same look from this distance. And so -- and so I probably stuck with that peak, and say, well, that peak looks -- of course, I'm already anchored on the internal standard, because I do know what that retention time is. And I then can identify that last peak, again, based on pattern, and these center peaks, based on pattern. And that's one of the ways that I would -- one of the ways that I would identify peaks, comparing them.

We have huge problems relying on this description. First, Dr. Brenna is NOT expressly describing what went on at LNDD. He's saying how HE would identify the peaks (presumably, based on the limited data he had in front of him -- he never said that this is the method he uses in his own lab). Plus, he's saying expressly that this identification method is NOT a method for measuring relative retention times. This is a problem, given that if LNDD had established IRMS "identification criteria", our best evidence is that this criteria WAS a relative retention time analysis. (Personally, I understand Dr. Brenna to say that the method he's describing is not the kind of relative retention time analysis performed by Dr. M-A; instead, it's a more relaxed and fuzzy kind of RRT analysis.) So, it's difficult to say that Dr. Brenna is describing the LNDD's identification criteria. (Again, I see this as proof that LNDD HAD no such identification criteria -- if they had such criteria, we should at least be able to figure out what the criteria was.) However, I have nothing else to go on. So ... in the absence of anything better, let's say that if LNDD had "identification criteria" as required under TD2003IDCR, this criteria was a relative retention time analysis, performed in the manner described by Dr. Brenna.

The next question I'll address here is: assuming the LNDD identification criteria are those described by Dr. Brenna, do they meet the requirements of TD2003IDCR. I think it's pretty obvious that they do not.

We can start with a definition of "criteria". My old Oxford-American dictionary defines "criterion" as "a standard for judgment." This is also the Wiktionary definition, more or less. So, a "criterion" is a kind of "standard". I'm not a scientist, but I think that the very essence of a scientific criterion or standard is that it allows for objective review and evaluation. A "criterion" has to be sufficiently descriptive and detailed to permit person A to determine the validity and accuracy of a judgment reached by person B. If a procedure to reach a judgment is subjective, so that two people can reasonably reach different conclusions while following the procedure, then the procedure is not a "criterion".

My definition of "criterion" is supported by a review of the procedures listed as acceptable criteria under TD2003IDCR. All of these criteria are detailed, objective and independently verifiable. There's little or no "wiggle room" under any of these criteria.

Now M, you may argue that it's difficult to create objective and scientific criteria for the visual comparison of two different graphs. It may be difficult, but it's not impossible. As an example, please take a look at WADA's own TD2007EPO, which contain objective criteria for the visual comparison of EPO results: you divide the result chart into three parts, you determine which two bands are most "intense" based on a densitometry measure, and so forth. It's an objective standard. You can use this standard to review the judgment reached by a lab that an athlete doped with EPO.

TD2007EPO is a good reference point for us to use when we look at the methodology for identifying compounds as explained by Dr. Brenna. Under the Brenna method, you identify a pattern of "large", "intermediate" and "small" peaks on each chart, and then you check to see if these two patterns are "approximately the same" from "this distance." Does the Brenna methodology come even remotely close to describing a "standard" that person A could use to check the work performed by person B? How do you distinguish a "large" peak from an "intermediate" peak? How do you determine whether two patterns are "approximately" the same? And what in the world did Dr. Brenna mean when he said that you'd compare the two patterns "from this distance"? How can you possibly use the Brenna methodology to review the judgment reached by a lab technician? You can't.

M, if you look at the Brenna methodology for identifying compounds, the methodology does not even remotely look like a criterion or a standard. As I said, I think that the best explanation of Brenna' testimony is that LNDD did not HAVE a standard for identifying IRMS compounds as required by TD2003IDCR, and that Brenna did the best he could under the circumstances to justify what LNDD did or failed to do. The only alternative to this conclusion is that the LNDD's method of identifying compounds is the one described by Brenna, and that this method fails miserably to meet the ISL requirement that labs have "criteria" in place for such identification.

Of course, we could also examine whether the Brenna methodology, which is the only description we have of the steps LNDD might have performed to identify the FL IRMS compounds, is an adequate methodology to measure the relative retention times that LNDD said it used (according to Ms. Mongongu and Dr. Ayotte) to reach its doping finding. According to the only evidence we have on how to measure RRTs, which is the testimony of Dr. M-A, the Brenna methodology is invalid. And Dr. M-A’s testimony here is supported by Dr. Brenna (as well as the majority decision), who proclaimed that it’s impossible to measure RRTs across two different GC machines. I think this is the point that tbv and others have made here: the methodology described by Dr. Brenna and endorsed by the majority decision is unsound as a matter of science. But from a legal standpoint, we can make essentially the same point, but in a simpler manner that I think is impossible to refute: whatever you might think of the scientific validity of the methodology described by Dr. Brenna (and OMJ), the methodology is not a “criterion”, and the methodology thus fails to meet the ISL requirements.

Under either alternative, we've proven a departure on the part of LNDD from the ISL, and we’ve proven that the majority decision was incorrect on this point. The next step will be to examine whether USADA effectively rebutted the presumption that this ISL departure caused the doping finding in the FL case, which is the only way left open at this point to defend FL’s doping conviction.

I'll pause at this moment for comments and to catch my breath.

m said...

Larry,

Don't have time to analyze this now. My suggestion polish this up, and ask TBV to post it, with a link to my post. Then we can all comment. This post is rapidly fading off the front page and nobody will read it.

DBrower said...

M,

Great minds sometimes think alike. I must have been posting the followup while you were entering your comment above.

Everybody, let's move on to this Post of Larry's comment

TBV

Unknown said...

This is an interesting discussion. Front page or not, I'll keep following it. Thank you for moving it up though. That will make it easier for all interested parties to locate.

Thanks!