An Idiot Looks at [Brenna 94]
Dr. Brenna was an author on a 1994 paper that has been cited variously in the case, both for and against Landis, by the usual suspects. Contributor Ali has gotten a copy, and files this evaluation...
By Ali
Curve Fitting for Restoration of Accuracy for Overlapping Peaks in Gas Chromatography/Combustion Isotope Ratio Mass Spectrometry
by Keith J. Goodman and J. Thomas Brenna
(Hereafter [Brenna 94])
(Hereafter [Brenna 94])
The purpose of this review is to summarize their findings and highlight any aspects that may have relevance to the matter at hand – the Landis case.
The background of the paper is that overlapping peaks in IRMS analysis result in inaccurate calculation of o/oo values, which we've looked at before. It says,
The conventional algorithm resulted in systematic bias related to degree of overlap
And it attempts to offer a new algorithm involving curve fitting to get better results. It also offers some experimental results on the affects of overlaps, which are of interest to us.
[MORE]
The conventional algorithm is separating the peaks with a vertical line at the centre of the valley between their overlap and taking that line down to what is assumed to be the background level. This is then used as an integration limit.
It's what our spreadsheet does, and appears to have been employed on a number of Landis’s IRMS F3 chromatograms to separate the 5B and 5a peaks from either themselves, or more frequently from some unidentified small peak that appears to be between and overlapping both the 5B and 5a peaks.
The paper gives a brief description is presented on the GC/C/IRMS. An integration time of 0.25 s was specified, which we believe to be analogous to the sampling rate. Peak start and stop are detected using only the 44 plot (due to superior signal to noise ratio). This differs from the LNDD process described by Brenna, where the 45/44 ratio plot is used to determine the start and stop times. Peak maxima are detected on each plot (44, 45 and 46) to identify the time shifts between the three detectors and the previously determined integration interval is applied to all three plots. Background is identified by a straight line fitted between peak start and stop points (which may be at the same level for constant background or at different levels for sloping background).
Due to the extra plumbing involved in the GC/C/IRMS process, both chromatographic efficiency and peak shape are detrimentally affected. In other words, you tend to get more peak overlaps and the peaks are generally not Gaussian, but are skewed (exhibiting an extended tail). Therefore, fitting a pure Gaussian peak to real data would not yield the best results.
To compensate for the inaccuracies involved in the conventional algorithm, the paper proposes to assess the ability of four different curve-fitting algorithms to recover the true peak shape and o/oo values of the overlapping peaks. What these four functions are may be of interest to some but aren’t relevant to this review.
Two substances exhibiting near identical o/oo values were used to experimentally generate a series of overlapping peaks. With equal sized peaks, at varying degrees of overlap (between 0% and 70%), it was observed that the conventional algorithm exhibited a depleted C13 ratio for the leading peak (-8 o/oo at maximum overlap) and an enhanced C13 ratio for the lagging peak (+8 o/oo at maximum overlap). This result was described as “unexpected”. The degree of error was proportional to the degree of overlap.
Brenna's testimony at the hearing leaned heavily on these measurements.
Application of the proposed curve fitting functions to the peaks yielded an improvement in peak area and o/oo recovery.
Similar experiments were run with a 10:1 ratio of peaks (leading peak ten times bigger than lagging peak). Using the conventional algorithm, at 40% overlap, detection of the smaller, lagging peak became problematic due to interference from leading peak’s tail. Beyond 40% overlap it was not distinguished as an individual peak.
The general trend of the leading peak being depleted and lagging peak becoming enhanced was observed for those cases where the two peaks were distinguishable.
With the peaks reversed and the leading peak being the smaller, the conventional method detected the smaller, leading peak at all degrees of overlap and it reflected a similar depletion trend as had been previously been observed.
The larger lagging peak exhibited very little error at all degrees of overlap. Curve fitting in all cases appeared to offer some advantages and generally improved the ability to recover the true o/oo of the peaks.
The curve fitting aspect is not strictly relevant to the Landis case, as it would appear that this has not used by LNDD. The conventional method and the results obtained are of more interest.
The first observation is that these results confirm Dr Brenna’s testimony of the leading peak’s C13/C12 ratio becoming depleted and the lagging peak’s C13/C12 ratio becoming enhanced. This contradicts Dr Meier-Augunstein’s testimony.
A significant factor effecting this contradiction is the effect of the m45 signal leading the m44 signal by approximately 150 ms.
Uncorrected, if the left-hand integration limit for the lagging peak is taken as the minima of the valley between the peaks and that is applied directly to both the m44 and m45 plots (not time shifted), then one would expect the C13/C12 ratio of the lagging peak to become depleted, having had a relatively larger proportion of C13 (m45) chopped off, compared to the slightly retarded C12 (m44) signal.
It would appear that performing the correction described by Brenna would resolve this issue but clearly it doesn’t. If we assume identical but scaled down peaks between the m44 and m45 plots, with no time shift (or a corrected time shift), the instantaneous 45/44 ratio would be homogeneous, presenting a constant value across the integration interval. In that case, the overlap of two peaks having identical o/oo values should have little or no impact on the measured value.
So why did overlap have such a significant effect in Brenna’s study and why were the results both contradictory to Meier-Augunstein’s opinion and described as “unexpected” by Brenna?
One possible explanation may lie in the integration time of 0.25 s.
In sampling at 0.25s intervals, how accurately will the peak maxima times on the m45 and m44 plots be identified? That’s what determines the required time shift so that they line up exactly. Remember that the required time shift is approximately 0.15s and we’re sampling at 0.25 s. A degree of error appears unavoidable. What if this error resulted in overcompensation for the time shift, resulting in a correction that had the m44 plot leading the m45 plot by some small but significant amount? This would reverse the effects described by Dr Meier-Augenstein and result in the observations made by Dr Brenna . [un-UPDATE: incorrection removed, don't ask. ]
To this idiot, this seems the most likely reason.
In a later post, we'll look at a paper in the GDC collection, in which Brenna looks at quantization errors. These are ones where you have insufficient or mismatched sample times.
19 comments:
Ali, have you seen the simulator at www.vias.org/simulations/simusoft_peakoverlap.html?
Word to the cautious -- that illustrates a simpler problem, the two peak with one sensor.
Our actual environment has two sensors (m/z = 44 and 45), making four peaks that overlap. It's way more complicated than presented by that sim.
TBV
Better yet...
The counter analysis of Iban Mayo's B sample will be carried out today in the French Châtenay-Malabry laboratory, regardless of whether the Saunier-Duval Prodir rider or a representative is present, according to todociclismo.com. The French laboratory received world wide attention after its practices were scrutinised during an American arbitration hearing over the positive test returned by Floyd Landis, who won the 2006 Tour de France before being striped of the title following the hearing.
Mayo tested positive for EPO on the second rest day of this year's Tour, with the A sample also analysed in the French laboratory. The B sample was sent to the University of Gent, in Belgium, as Châtenay-Malabry closed its doors for holidays, but the B sample analysis gave a negative result. An analysis completed in Sydney also found the samples to be negative, which prompted the Spanish Cycling Federation (RFEC) to declared Mayo innocent.
RFEC requested that the case be closed, however the sport's international governing body, the UCI, didn't trust the Belgian analysis, which has lead to today's the counter analysis in France. Mayo has reportedly objected to the counter analysis and has sent various letters of protest to the UCI.
The UCI's anti-doping chief Anne Gripper has said the counter analysis will go ahead today, even if neither the rider nor any of his representatives will be present.
WTF. Why do they even have rules???
UCI's Anne Gripper has no compunction about violating "the rules" in her pursuit of dopers. After all they made the rules, mostly as a public relations gimmick, so they can change or abrogate them at any time, especially since there isn't anybody to hold them accountable.
Michael,
WTF indeed.
WTF and WAF (what a farce)
Ali, two questions: did you switch names in the last sentence of that look at Brenna 94? I thought I understood what you were saying, but then the last line seemed like the names were backward.
Also, any comment on how Brenna's study would change if we were not talking just about two overlapping peaks, but that small third peak between them (especially if it had a highly negative CIR)?
syi
David,
To follow up Mike syi's comment above, is it possible to get the missing GDC docs? That would be those in the GDC 00921-00980 gap. It would be nice to see the 2/1 traces for the third fractions. They might give us a clarification on the question syi has, what happened to the small interstitial peak in the stage 17 third fraction. Right now we only have the stage 17 b sample reprocessing docs for the first and second fractions.
Ferren
syi,
I was trying to say WMA claimed that the lagging peak would have its C13 ratio depleted, assuming that the m45 signal leads the m44 signal. If the data processing in Brenna's study reversed that and had the m44 signal lead the m45 signal, the effect on the lagging peak would change to enhancement, as Brenna observed. I must admit a disjointed shift in focus at the end, though.
I think my primary comment on your second question is that the presence of a third small peak between two peaks which already exhibit overlap will further distort the apparent point at which your leading peak ends and your lagging peak begins, thus throwing your integration limits out. Determining what is background and what isn't will also become more difficult. I think this is evident in the Landis F3s where in some instances they have separated peaks with a "perpendicular drop" (vertical line down to assumed background level), whilst in other cases they have removed background as though it was sloping (those F3s where you see a sloping line at the base of the peak). If you then assumed a highly negative value for your small interfering peak, then I think the effect may depend on many factors, including background removal strategy, degree of overlap, relative magnitude, etc. Specifically, depending on the degree of overlap, you may or may not identify your interfering peak as a distinct peak. If you could, then you may be able to minimise it's effect by chopping it out from between your obverlapping peaks. Basically, I don't think there's any one answer except maybe "it depends".
Ali
It was great to see that Brenna used the m=44/45/46 time shifing approach to the isotope ratio analysis. I thought that approach might have merit in reducing some of the errors that the spreadsheet demonstrates. (See earlier comments.)
His results that show that the trailing peak has an elevated isotope ratio might be explained by too much of a time shift. Aligning the peaks of the three plots might not produce the optimal alignment. It appears that the m=45 peak should be slightly earlier than the m=44 peak for proper alignment.
If the alignment needs to be finer than 250ms, then it should be possible to calculate a weighted skew that would be a good approximation. For a measure 50ms earlier, take 20% of the 250ms earlier measure and 80% of the current measure.
I earlier suggested a center of mass type approach. I would guess that in the Brenna data, the peaks do not align with the center of mass. The center of mass approach might be more resilient to non-linearities with respect to peak height and bi-lateral symmetry. The center of mass approach will also calculate a time alignment that is finer than 250ms.
At 7:01 Mike wrote:
any comment on how Brenna's study would change if we were not talking just about two overlapping peaks, but that small third peak between them (especially if it had a highly negative CIR)?
Some idiot answered this a few days ago:
Dr. Brenna said much the same thing in his 1994 study of peak overlap:
"Traditional chromatograms may be thought of as a collection of well-resolved singlets with a few doublets of varying degrees of overlap. The recovery of information from doublets is the subject of this work. It is assumed that triplets and higher multiplets, although rare according to theoretical considerations, will be resolved chromatographically."
So, if we have a triplet, as we do in the F3 GCMS chromatogram for 5bA, it's YGIAGAM, and you have to do the separation over again and try to get better peaks.
Seems he forgot about this back in May.
A memory is a terrible thing not to have!
TBV
Well excuuuuse me!
I was hoping for a little more analytical response from our kind host from the Golden State, and our keen friend from the land of haggis.
But if YGIAGAM is the best response, so be it.
syi
I think YGIAGAM is the best and only response to your question.
I've read many articles on this subject now and a recurring theme in every one is baseline separation of peaks is of paramount importance to determining the true o/oo value of a peak. That was the raison d'etre of Brenna's paper on curve fitting. His curve fitting algorithms appeared to minimise the error created by overlapping peaks, but not remove it. LNDD didn't even try to minimise it.
We're talking about what degree of error can we recover accurate results from. Brenna's paper concluded that using the conventional alogorithm, the answer was practically none and that was with two known substances having equal delta values !. If they can't perform the required chromatography to yield accurate results, they'll never get accurate results and they shouldn't be using it as evidence.
Let's not forget that LNDD claim +/- 0.8 o/oo accuracy. Does anyone here believe that to hold true when peaks of unknown delta values overlap ?. Surely the research indicates that not to be the case ?
This is just a complete farce. Any pretence that this case was being judged on scientific principles should have been dropped a long time ago. This is just about saving face for the lab and the ADAs.
Ali
Ali,
How does Floyd prove what you said in the CAS hearing?
Mike
Mike,
I'm not sure which bit you were referring to, but I'm going to assume it was the +/- 0.8 o/oo accuracy of the lab because this one could have some legs.
I imagine that this accuracy is quoted for well separated peaks, not just any possible scenario (eg a forest of interfering peaks). Brenna's paper established that error, over and above the basic tolerances of the equipment/process, are introduced when peaks overlap. There are numerous other studies which support this.
If it can be demonstrated that the LNDD tolerance of +/- 0.8 o/oo would, in all likilehood, be exceeded in the presence of overlapping peaks, would that mean that their results do not meet their own criteria for accuracy ? It would be difficult to establish what the error band is ... +/- 1, +/-2, +/-4 o/oo ?.
The thing is, does it matter ? If they didn't meet the required level of accuracy, those results obtained when peaks overlap are invalid or are of unknown delta values.
It's certainly a genuine observation and one which may be relevant.
Ali
Okay, YGIAGAM is it then. In that case, have we already gone the following:
In the GCMS chromatogram there is clearly a small peak between 5bA and 5aA. But it is not as clear that there is something between them in the IRMS. There is the lagging tail of the 5bA and a small "ramp" leading up to the 5aA, but is there any indication that that peak is still there. They can completely disappear in the IRMS, as that one obvious one that eluted earlier in the GCMS, but was completely gone in the IRMS.
Is there any way to settle this? Or is speculation on all sides?
syi
syi,
Looking through the F3s, it appears that in some cases they managed to identify this small interfering peak and chopped it out with the integration limits. However in other circumstances, they appeared to include it and in yet another variation, they chose to consider it as sloping background. How gloriously inconsistant !. I suppose whether they identified it as a separate peak or not depends on the state of their highly calibrated "eyeball".
Not realated to your question, but now a growing obsession for me ... the +/- 0.8 o/oo accuracy. There's a falacy here which has been propagated by all concerned at LNDD. That number represents the limitations of the equipment and process used to generate the IRMS data. It doesn't account for the additional errors introduced when processing that data, especially in the case of interfering peaks (does it ?). In fact, by implication of their threshold for doping being -3.8 delta, they are saying that processing ovelapping peaks contributes NO ERROR, even though they adopt no curve fitting to compensate for the INEVITABLE ERROR that Brenna observed.
Curious ...
Ali
"Looking through the F3s, it appears that in some cases they managed to identify this small interfering peak and chopped it out with the integration limits."
First, on what basis are you claiming that the "small interfering peak" is even there in the IRMS? I forget, is it more visible on the stages other than stage 17?
Second, if it is, how can you know where, in fact, they placed the integration limits?
syi
Syi,
USADA0172 – IRMS analysis of Stage 17 F3. USADA0173 – The corresponding plot. No obvious middle peak, however, zoom in on plot and you’ll see the integration limits. The start of the peak appears to be indicated by the long dash, the end appears to be a dotted line. This pattern appears consistent across all measured peaks on plot. The 5a appears to be either on a sloping background (if you ignore the marked up integration limits) or has the tail of the 5B overlapping. There’s no indication of how they removed the background, but from USADA0172, they say the background is approx -50 o/oo. In other words, there’s a decision to make on the 5a background removal because the start of the peak is higher than the end and that background is very negative. Leave even a trace of background in your selection and you’ll skew your result in a negative direction. It’s OK though, LNDD never exceed +/- 0.8 o/oo.
The other’s I refer to are exhibits 86 to 92 in the Nov major doc release. These plots differ from the earlier ones in that the background removed appears to be indicated on the plots and there appears to be more evidence of an extra interfering peak between 5B and 5a, eg Ex 86, page LNDD0894 – Landis F3, sloping background route. You’ll see a variety of approaches to removing background/middle peak through these exhibits. It’s OK though, LNDD never ever exceed +/- 0.8 o/oo. Armed with that knowledge, it’s no wonder they like to get creative with their background removal. It must add a bit of excitement to an otherwise dull job.
I’d have hyperlinked the above references but my strategy of using the slipstream effect to keep up with technology has proven to be disastrous.
Ali
Post a Comment