Friday, May 25, 2007

You3 does some 'splaining

In this post by You3 at DPF, we get the start of an understanding of the impact of background subtraction, along with a summary of the important parts of the defense as presented:

I think that the Landis defense was pretty clear that they assert that there was an error in the analysis.

The defense lawyers presented evidence that

1) the analysis failed to conform to various International Labratory Standards,
2) the analysis failed to conform to the IRMS equipment manufacturer's specifications,
3) the documentation on what algorithms are implemented for background estimation and subtraction by the IRMS machine no longer exist
4) the operators and their supervisor were not trained by the IRMS manufacturer or someone qualified by the manufacturer
5) the executed process was not documented sufficiently to be reviewable or repeatable,
6) there were no controls in place to establish or validate the accuracy of background/baseline subtraction for complicated urine matrixes
7) there were no IRMS calibration samples for 3 of the 6 metabolites
8) there were no SOPs in place to specify the process for the placement of integration limits.

[more]


From here on out, we're starting to get into the deep end.

I don’t think this forum [DPF] has done much exploring the process of background subtraction and the nature of the potential errors that might occur. As far as I could tell, the Isoprime1 software allows the specification of some number of points that are believed to lie on the the background baseline curve. The software then interpolates a curve between the points and considers this the background baseline 12C height. Further, it must interpolate the d13C values along the line between the points. This, in effect, establishes a 13C height curve, but the height is 2 orders of magnitude smaller than the 12C curve. To the extent that the reference points are further apart or the actual background baseline is more complicated, the more prone to error will be the estimate of the curves.

The testimony from Davis was that no one knows anymore what algorithms are used by the IsoPrime1 OS2 software to do the interpolation of the 12C background baseline height or the d13C values along that line. Anyone who has drawn smooth curves with reference points in PowerPoint or any other drawing program knows that unexpected twists in the curve can occur as points are specified. The IsoPrime1 operator could view baseline curve and the effect of the baseline curve on the m/z 45/44 ratio plot. But, there was not indicatation that the user interface provided explicit information as to the quality of the fit of the curve, although it did provide the d13C measurements of the peaks after background subtraction.

Let’s look at how sensitive the d13C measurement is to accurate background baseline subtraction. It turns out that the smaller the elute peak and the larger the background contribution, the more sensitive the measurement is to errors in the background measurement. By my estimate, the sample B 5a-diol (USADA0350) measurement is 4 times more sensitive to background measurement error than the 5b-diol measurement in “Mix Cal Acetate 001A-100ng ing” (USADA362). Error is also more likely because the background curve is complex and there appears to be co-elution in the neighborhood of the 5a-diol peak. This differing sensitivity to error between MixCal and urine samples tends to undercut the assertion that the MixCal analysis establishes linearity of measurement since the measurement of background ratios would tend to stress the linearity of absolute measurements of m/z 45 and 44 quantities. In particular, the background has the lowest currents (abundance), but the measurement must still be accurate.

The background area underneath an elute is not documented in the LDP. In order to estimate the area, I assume the elute peak is a Gaussian curve. From the area and the height, the standard deviation can be calculated. I then estimate the width of the peak as 6 standard deviations. I estimate the height of the background contribution from looking at the plots on USADA0349 and USADA0363. Then, I calculate the area of the background as the product of that height and the width of the elute peak.

The table below captures the characterization of the two peaks:
(edit: Note, I take the reported area to reflect just the elute, not inclusive of the background. I can't be certain this is correct. By examination of the plots, the height includes background.)


5a-diol 5b-diol
Landis B/F3 MixCal
Values from the LDP:
Elute d13C: -31.88 -33.63
Raw peak height: 2.33e-09 3.28e-09
Elute 12C area: 2.1502e-08 2.7273e-08
Background d13C: -56.94 -77.27

Estimates:
Background height: 5.0098e-10 2.0077e-10

Calculated values:
Elute std dev 4.69 3.53
Percent raw 12C
area elute: 60.4% 86.5%
Percentage raw 12C
area background: 39.6% 13.5%
13C/12C ratio: 1.087896e-02 1.085929e-02
Elute 13C area: 2.3392e-10 2.9617e-10
Elute width: 28.1 21.2
Elute height: 1.83e-09 3.0792e-09
Raw peak d13C: -41.80 -39.52
Raw peak 13C/12C: 1.076744e-02 1.079309e-02
Raw peak 12C area: 1.4097e-08 4.2565e-09
Raw peak 13C area: 1.4939e-10 4.4135e-11



I can now calculate the amount of background 13C that needs to be attributed to the elute peak, instead of the background, in order raise the d13C measure of the peak by 3 delta units. The Landis 5a-diol needs to change attribution of 0.48% of the background 13C, while the MixCal 5b-diol needs to change by more than 4 times more at 2.01%. Note that the background d13C measure for the Landis 5a-diol is still in line with that of the blank sample analysis at -64.54.

If the background subtraction for the Landis sample was in error, attributing 0.48% too much 13C to the background, then a d13C measure that was 3.00 delta units too negative would result. Did this happen?



5a-diol 5b-diol
Landis B/F3 MixCal
Percentage of
background 13C
re-attbributed to
elute: 0.48% 2.01%
Percentage of
elute & bkg 13C: 0.190% 0.271%
New elute 13C area: 2.3465e-10 2.9709e-10
New elute 13C/12C: 1.091283e-02 1.089311e-02
New elute d13C: -28.87 -30.62
Delta change: 3.01 3.01
New bkg ratio: 1.054569e-02 1.015224e-02
New bkg d13C: -64.54 -96.55


Lastly, I calculate the percentage of background 13C that needs to be attributed to the elute in MixCal analysis in order to generate a 0.5 delta error, such that the error would be at the edge of the margin of uncertainty. When this percentage of re-attribution is applied to the Landis sample a delta change of 2.01 occurs, an error more than 4 times larger.


5a-diol 5b-diol
Landis B/F3 MixCal
Percentage of
background 13C
re-attbributed to
elute: 0.3341% 0.3341%
Percentage of
elute & bkg 13C: 0.1323% 0.0451%
New elute 13C area: 2.3443e-10 2.9632e-10
New elute 13C/12C: 1.090254e-02 1.086492e-02
New elute d13C: -29.78 -33.13
Delta change: 2.10 0.50
New bkg ratio: 1.056138e-02 1.033284e-02
New bkg d13C: -60.14 -80.48


This analysis does not consider the contribution of oxygen isotopes to the errors. The background curve interpolation actually needs to interpolate a baseline value for all three detected ions. Around 7% of the m/z 45 ions are actually oxygen isotopes, not 13C. The contribution of the oxygen isotopes is calucated from the abundance of m/z 46 ions.

Any error that creates a more negative measurement is amplified by a reduced derivitization ion correction. An error of -3.0 delta units will generate a larger error in the corrected value.

18 comments:

Anonymous said...

I can't claim to understand, but I will make the statement that You3 is one very smart person. I nominate You3 for T-3's job at USADA when the axes start to fall.
MAN

Anonymous said...
This comment has been removed by a blog administrator.
Daniel said...

You3,

I have always loved the consistent impartiality of your writing, and can only hope that one day your hard work will be a major reason why Floyd will eventually be proven to be telling the truth -be it in 2010 or 2020.

Anonymous said...
This comment has been removed by a blog administrator.
Anonymous said...

Yeah, pretty much lost me at 'deep end,' but I think I understand that 0.47% error in baseline subtracion could result in a -3.00 difference in the result in the 5a metabolite that is at issue. How significantly does a lab tech clicking new data points have to be off to make a 0.48% error, and is there a way such an error could be systematic to the methods of the lab techs (esp. given that one "trained" the other)? Please let me know if I am massively ovesimplifying.

bamalaw the math-tard

john murray said...

I've been following this with more and more outrage . . .

I've been developing software since the mid 80's and have serious questions. Is the analysis software available? Virtual Machine software (Virtual PC, VM-Ware) is easily capable of hosting older operating systems, including O/S-2 on modern platforms. It would be very interesting to see the "results" reported by the paris lab for ourselves . . .

Anonymous said...

John Murray - I believe the lab reports are here http://trustbut.blogspot.com/2006/10/lab-documentation-package.html

Anonymous said...

From this post: "Since LNDD has 3 times more AAF's from IRMS than the other labs, I would like to know what percentage of those are from excessively negative 5a. I have no idea what they would be doing to have systematic problems specifically on the 5a, but to this non scientist they lost me when they had one of their urine blanks as a doper in the 5a. Granted it was only -3.5? but on the blank??????"

Am I to understand that one of the control samples (ie. not Landis) came back positive?

john murray said...

thanks for that, I was referring to the EDF's - the raw data files processed by a 3rd pary.

This is *exactly* what bothers me so much about the process - science demands raw data, wada/usada insists on hiding it - we are left with photocopies of scrawled "conclusions".

Anonymous said...

Frankly, the science of this is beyond me, as I think it is for most everyone that posts here or at DPF. As someone else noted, it would have been nice if the media had employed a real scientist to attend the hearings, rather than sports people, who just wrote about how "boring" the science was. At the moment, though, it seems that the only scientist who matters is Botre. From what I can gather, there are reasoned scientific explanations of how any lab errors still could not have caused false positives for Landis, and so I don't think it's a foregone conclusion he will win.

The other thing I find interesting the apparent shift of opinion on this site from Landis simply being innocent to - he may have or even probably doped, but the lab was so bad he should be found innocent. I think if Landis had thought about things a little more, maybe he would have chosen a private hearing, as I think his reputation is now shot one way or the other, which I'm sure is not what he wanted from this process. Also, for whatever he did prove, he didn't prove all the stuff he said he was going to prove, and this makes him look bad too. It will be interesting to see who next asks for a public hearing.

And, along the same lines, does anyone have any information on how the Landis camp is now viewing the results of the hearing, and if there are going to be any changes -aside from Will G. - to the Landis camp. From what I can gather, some of those involved were at least initially agaisnt a public hearing, for example, while others pushed for it. It may be too late to change courses now. And in that regard, is a CAS hearing by rule open or closed, or is it up to the parties?

Anonymous said...

You3,
I'm curious what your background is (Analytical chemist, perhaps?)? I work in an FDA regulated lab, but I am in product development, not analytical chemistry. After seeing that LNDD used white-out in the Arnie Baker slide show, I was not suprised that the defense found sytematic, lab-wide failures. Given this, I'm curious what your thoughts are in regard to the lab results overall.

Regards,
Bob/Phoenix

cam said...

Anon 6:36 -- i believe the CAS hearing is closed, having nothing to do with the participants' choice.

i do disagree with you that Floyd was wrong to have these hearings open. i think it was eye-opening to say the least at the shoddy lab practices at LNDD. is this where the 2007 TDF urine samples are going? it makes one shudder.

i respect that Floyd left it open for all to see. yes, he exposed himself but what of it? oh my god, we found out he's actually human (the post on DPF). the only thing that really makes anyone cringe is the infamous Call and that was done by Will, not Floyd, and could *never* have been anticipated. to be perfectly honest, i have even more respect for Floyd because he made this open to all. it took, perhaps, even bigger cajones than Stage 17.

bravo!

Anonymous said...

I don't dispute that the open hearing was beneficial as regards the lab procedures, but my question really was more directed to Landis himself. He seems to have brought himself down along with the lab - just read the general press - in fact, read just about everything except what's posted here. I just can't believe Landis now thinks it was in his best interest to have the public hearing. I guesss time will tell, and we'll be able to tell by how the FFF does now.

pem said...

Anonymous 11:20 AM:

I disagree. I believe Landis has benefited greatly by making his hearing public.

The mainstream media only reports what is sensational. Most of the public listens only to the mainstream reports. If Landis had kept everything private, then that is all everyone would know.

However, by making it public, Landis has a number of followers that are very interested in the process – some are cycling fans, some are scientists, some are lawyers, some are competitive athletes, etc. No matter the outcome, these witnesses are seeing how unfair and corrupt the current system is. And it is a group of smart people that are seeing this.

Based on the comments I have been reading, Landis has the right minority of people supporting him in his plight.

Anonymous said...

Does anyone else find it ironic that Floyd's case, which is being prosecuted by the United States Anti Doping Agency before the American Arbitration Association is going to be decided by four judges (yes, I'm including Botre) only one of whom is actually American? Campbell is the American.

Your hard earned tax dollars at work.

Anonymous said...
This comment has been removed by a blog administrator.
Anonymous said...
This comment has been removed by a blog administrator.
Anonymous said...

If Landis had gone with a closed hearing then none of us would have been exposed to the quality of the work done at LNDD or the closed ranks approach of the WADA certified labs.

Taking it public revealed the incompetence and the stacked deck that could make it a winning hand.

The reason that folks' attitude changed is that they got educated on the sloppy lab work and the fact that the samples were gone and no further testing of them could be done. With better understanding of the process they became aware that the "best" outcome was for Landis to be able to destroy the credibility of the test results.

Doing that would not prove that he did not dope and people realized that. The opinion that readers will have of Landis will have to be based on belief in his character and credibility. There may be other old samples out there that could be run through competent labs, but those that believe he doped will discount negatives (masking, not doping at the time, etc.) and carry their opinion forward.

We need a site independent of FFF to serve as a clearing house/ repository for sound criticism of the system - Judge Hue's early analyses, the Law refiew article, transcripts of testimony, articles or opinion pieces by technical people who are willing to put their names to them - that can be drawn on and presented to people and organizations that can effectively act.
pcrosby