tag:blogger.com,1999:blog-31819641.post718988205048349767..comments2023-10-06T03:21:26.130-07:00Comments on trust but verify: Idiots Return to Brenna and WM-A, Part IIDBrowerhttp://www.blogger.com/profile/17718913310467614671noreply@blogger.comBlogger17125tag:blogger.com,1999:blog-31819641.post-30864060998175396582008-01-30T04:49:00.000-08:002008-01-30T04:49:00.000-08:00Larry,You're confusing me with your references to ...Larry,<BR/><BR/>You're confusing me with your references to "automatic" and "OS2" as though they are different things. The OS2 can operate in either manual or automatic modes. On top of that, we also have the Masslynx results.<BR/><BR/>Yes, if you have some m44-m45 misalignment, placement of the integration limits is important with regard to how big the error is, but I'm just trying to establish that you get an error with m45-m44 misalignment and its direction depends on which leads which.<BR/><BR/>I'm trying to establish that Brenna's and Meier-Augebstein's predictions on direction of error are both equally possible and it depends on the system you're using.<BR/><BR/>Let's see if we can agree on that before complicating the matter with less simple scenarios than the ones used in my examples.Alihttps://www.blogger.com/profile/00786387057717601356noreply@blogger.comtag:blogger.com,1999:blog-31819641.post-81270935555315809652008-01-29T17:29:00.000-08:002008-01-29T17:29:00.000-08:00Alaisdair and wschart -Alaisdair, you said: when y...Alaisdair and wschart -<BR/><BR/>Alaisdair, you said: <I>when you have perfectly aligned (and perfect) m44 and m45 peaks, it doesn't matter a damn where you place your integration limits.</I> Yes, I realize that. You might then test centroiding by moving around the integration limits to see if it affects the CIR results. If it does, then you haven't achieved perfect centroiding of perfect peaks. <BR/><BR/>But you're not going to have perfect peaks. I think that's where this gets complicated. Say that the m44 peak is a perfect peak and the m45 is not perfect. Say that the m45 is "front loaded" -- the slope going up is steep and the slope going down is more gradual, sort of like the left-most two strokes in a capital "N". In such a case, it matters a lot where you put your integration limits, because if you put your beginning limit a little too far to the right, you're going to cut off proportionally more m45 than m44.<BR/><BR/>As far as the methodology you use to set integration limits ... you spent a whole month of your life convincing me that this is not an easy thing to do! Again, if we have two overlapping perfect same-sized m44 peaks, then sure, you'd put the integration limit at the lowest point where the two curves intersect -- as you said, where else would you put it? I'm not so sure we could be this confident if we were dealing with two imperfect overlapping peaks of different sizes. Imagine for the moment that we have two m44 peaks both shaped like the left-most two strokes of a capital "N". Make them the same size, and draw them so that they overlap. Then draw your integration limit at the low point of the intersection. If you do this, you'll end up with a lot of the left-most peak to the right of your integration limit, and very little of the right-most peak to the left of your integration limit. That's probably not a good thing. <BR/><BR/>I think you hit the same problem with perfectly shaped peaks of different sizes. These perfectly shaped peaks have bell-shaped curves, meaning that their slopes change as you move from beginning to end of the curve. The steepest slope on the curve will take place around the half-way point between the curve beginning and where the curve reaches its top. A small curve will presumably have a smaller RT to reach this steep point than a larger curve. I'm no geometrist, but it would seem to me that at the point of intersection between a perfectly shaped large curve and a perfectly shaped small curve, the slope of the small curve will be relatively steep and the slope of the large curve will be relatively flat. So if you drew your integration limit at the low point of the intersection of these two curves, you'd have the same situation I described above with curves shaped like the first two strokes of a capital "N" - you'll have a lot of the left-most peak to the right of your integration limit, and very little of the right-most peak to the left of your integration limit. That's probably not a good thing either. <BR/><BR/>And I'm not even considering most of the problems you discussed back in November, like noise levels and sloping baselines.<BR/><BR/>Now, I don't know whether any of this affects where a lab would set its integration limits. Maybe there isn't any other place to put an integration limit except at the lowest point of intersection, at which point you close your eyes hold your nose and hope for the best. But it still seems to me that there is some potential for error introduced here, and that this potential for error might be as significant as the error introduced by poor centroiding.<BR/><BR/>On to centroiding. My guess is that either the LNDD software does not compensate for the m44-m45 time difference, or that it does so in a relatively simple way (for example, by setting the m45 integration points 150 ms to the left of the m44 integration points set by the operator). The LNDD software is close to 20 years old, and I doubt it could do very much more than what I'm describing. Consider that the LNDD's IRMS was so old, they could back up its hard drive once a year on a single CD-ROM. That doesn't suggest a lot of graphical processing ability. I'm also guessing that when the EDFs were reprocessed using newer software, the newer software included a newer (hopefully better) method for centroiding. So I don't assume that the manual and automatic processing of the EDFs used identical centroiding.<BR/><BR/>If the old LNDD software did not do any centroiding, then all other things being equal, it should have produced more negative results than the automatic processing (which we presume DID employ centroiding). Of course, we saw the opposite result. It's difficult for me to interpret this finding the same way you have, but also, it's difficult for me to interpret this finding in a way that makes any sense whatsoever. My assumption is that the automatic processing would do a better job of centroiding than the OS2 software (which may not have even PERFORMED any centroiding). So why, then, did the automatic processing appear to have reached results that appear to be too negative? Maybe the LNDD technicians compensated for the poor centroiding performed by the OS2 software by setting the integration limits to make results less negative (and since we're dealing with imperfect peaks of different sizes, I'm assuming that it may be possible to do this) ... but that still doesn't explain why the automatic processing results seem so far off.<BR/><BR/>Perhaps the answer lies with Mr. Idiot's recent discovery of a connection between column polarity and m44-m45 offset?Larryhttps://www.blogger.com/profile/08976868079076669453noreply@blogger.comtag:blogger.com,1999:blog-31819641.post-86021248294997325572008-01-29T06:28:00.000-08:002008-01-29T06:28:00.000-08:00Larry:RE "integroiding": once the limits of integr...Larry:<BR/><BR/>RE "integroiding": once the limits of integration are set, by whatever method, the integration itself is a rather straightforward, if perhaps complicated, mathematical calculation. Any degree of inexactitude (if that a new word?) comes from the estimation of the limits, the alignment, and any other "fiddling" is done with the raw data;wscharthttps://www.blogger.com/profile/14580006249706915137noreply@blogger.comtag:blogger.com,1999:blog-31819641.post-61924806306936457192008-01-29T02:06:00.000-08:002008-01-29T02:06:00.000-08:00Larry,With regard to: "The point I was trying to m...Larry,<BR/><BR/>With regard to: "The point I was trying to make is that the CIR methods would all involve centroiding followed by setting integration limits ("integroiding"?). Both centroiding and integroiding introduce the possibility for error. If you're looking at a CIR result that's too negative, you can't necessarily say it was caused by faulty centroiding. If you're comparing the LNDD results to the results you see in Dr. Brenna's paper, you can't necessarily say that Brenna centroided to the right and LNDD centroided to the left, unless you know that they both used identical integroiding."<BR/><BR/>Let's go back to Brenna and forget about LNDD for the moment. Part 2 showed (perhaps not explicityly) that when you have perfectly aligned (and perfect) m44 and m45 peaks, it doesn't matter a damn where you place your integration limits. If you chop off part of your peak, you're losing m45 and m44 in equal proportions - you havn't changed the ratio. If you include part of another peak in your integration interval and that peak happens to have the same CIR as the peak you want to integrate (a la Brenna), that doesn't matter either, because the m44 and m45 you include from the other peak is in the same proportions as the peak you're integrating - you havn't changed the ratio.<BR/><BR/>So, when Brenna observes a systematic error, integrating overlapping peaks of identical CIR, I can say catagorically that, for whatever reason, his m44 and m45 were not perfectly aligned. I can also say that m45 lagged m44, because that's the only way you can get a systematic an error which makes your peak look less negative than it really is. I think if you study the diagrams in Part 2, you'll see what I'm talking about.<BR/><BR/>He was quite explicit in the methodology used to set the integration limits (although I read that as him describing what the software did, not what he did). With just two peaks overlapping (and say you're interested in the right-hand peak) you choose the minima at the intersection of the two peaks for the left hand limit and somewhere else which includes the rest of your peak for the right-hand limit (see Part 2). To be frank, where else would you place your limits ?.<BR/><BR/>Now to the centroiding factor. My understanding is that if this is done, it is done by the software, not the operator. In manual mode, they may be able to set the integration limits, but they don't align the m44 and m45 peaks. That's done by the software. It would be virtually impossible for that to be done accurately by eye.<BR/><BR/>Now back to LNDD. We don't know if the OS2 software used to analyse the Stage 17 results even compensates for the m44-m45 time difference. It's so old, nobody seems 100% sure what it does. Maybe it does and maybe it doesn't. If we knew for a fact that it didn't, we'd know for a fact that it will bias results in a more negative direction when you have overlapping peaks (as WM-A testified). If we knew for a fact that it did, then any misalignment between m44 and m45, from whatever source, would be altered by the software in an attempt to correct it. In Part 2, we made no assumption either one way or the other. We tried to deduce from what happened with the blank during the reprocessing of the Stage 17 EDFs.<BR/><BR/>PS: I'm pretty sure "integroiding" is not a word I've ever heard of :-)Alihttps://www.blogger.com/profile/00786387057717601356noreply@blogger.comtag:blogger.com,1999:blog-31819641.post-26063929006527038272008-01-29T00:30:00.000-08:002008-01-29T00:30:00.000-08:00syi,Excuse me while I slap my own forehead !syi,<BR/><BR/>Excuse me while I slap my own forehead !Alihttps://www.blogger.com/profile/00786387057717601356noreply@blogger.comtag:blogger.com,1999:blog-31819641.post-8798961585696505802008-01-28T18:46:00.000-08:002008-01-28T18:46:00.000-08:00Well I didn't read the whole book. Just WMA's art...Well I didn't read the whole book. Just WMA's article/chapter 8. Funny how little stuff sticks in your brain.<BR/><BR/>syiMike Solberghttps://www.blogger.com/profile/11784753552166129987noreply@blogger.comtag:blogger.com,1999:blog-31819641.post-88227883842106363712008-01-28T18:07:00.000-08:002008-01-28T18:07:00.000-08:00Wow. I don't know whether to be more impressed wi...Wow. I don't know whether to be more impressed with the point you're making, or with the fact that you made it all the way to page 168!<BR/><BR/>syi, I think this could be a very, very important point.Larryhttps://www.blogger.com/profile/08976868079076669453noreply@blogger.comtag:blogger.com,1999:blog-31819641.post-53619134577356390432008-01-28T17:26:00.000-08:002008-01-28T17:26:00.000-08:00Come on guys, I don't just make this stuff up! Yo...Come on guys, I don't just make this stuff up! You don't trust me? Me, Mr. Idiot?<BR/><BR/><A HREF="http://www.amazon.com/gp/reader/0444511148/ref=sib_dp_pt/102-6969289-9241756#" REL="nofollow">here it is</A><BR/><BR/>Search "polarity" and go to page 168.<BR/><BR/>It's got to do with Van der Waal's dispersion forces. Duh.<BR/><BR/>I think I read the full article months ago at my local med school, but, in any case, this is enough to see the comment.<BR/><BR/>syiMike Solberghttps://www.blogger.com/profile/11784753552166129987noreply@blogger.comtag:blogger.com,1999:blog-31819641.post-19933296868081238142008-01-28T16:52:00.000-08:002008-01-28T16:52:00.000-08:00TBV -Maybe someday we'll talk about noise. I'm pr...TBV -<BR/><BR/>Maybe someday we'll talk about noise. I'm probably not up for a discussion about linearity. I just want to know what the noise IS. Are we looking at the echo from the big bang? Is this just a low level stream of undifferentiated stuff that didn't get filtered out by the sample preparation or clumped into peaks by the GC?<BR/><BR/>But we won't talk about noise today.<BR/><BR/>The point I was trying to make is that the CIR methods would all involve centroiding followed by setting integration limits ("integroiding"?). Both centroiding and integroiding introduce the possibility for error. If you're looking at a CIR result that's too negative, you can't necessarily say it was caused by faulty centroiding. If you're comparing the LNDD results to the results you see in Dr. Brenna's paper, you can't necessarily say that Brenna centroided to the right and LNDD centroided to the left, unless you know that they both used identical integroiding.<BR/><BR/>So while I see the potential for error, I don't see it pointing in a particular direction. Yet.Larryhttps://www.blogger.com/profile/08976868079076669453noreply@blogger.comtag:blogger.com,1999:blog-31819641.post-21989472655375364962008-01-28T15:32:00.000-08:002008-01-28T15:32:00.000-08:00When you start talking about noise, you run perilo...When you start talking about noise, you run perilously close to talking about linearity at the low end of the detection capabilities.<BR/><BR/>I don't think we want to go there in this particular series.<BR/><BR/>It's probably worth talking about the joint or independant setting of the m44 and m45 integration limits.<BR/><BR/>Our understanding, acquired after the original idiots series, was that practice in the field is to to a time adjustment of one or the other, often using "centroiding" as the mechanism. The very use of the -oid suffix should give some clue that the technique is an approximation, not exact.<BR/><BR/>Our contention is that this works fine for non-overlapped peaks, where the errors don't much matter. It's when they do overlap the errors in the algorithms start to matter. Shade one way, one result; shade the other, opposite result.<BR/><BR/>This could be one of the things that counted in the "explain what the machine does for integration" discovery questions that could not or would not be answered.<BR/><BR/>What is important is that it is a systematic, systemic, likely to be consistent error that appears not to be accounted for -- and which is likely to apply to all of the overlapping 5bA and 5aA peaks that LNDD tests, including the blanks, and all the Landis samples. Depending on the overlap, the results may skew more.<BR/><BR/>It raises questions again about the much higher rate of testosterone positives found by LNDD than other labs.<BR/><BR/>TBVDBrowerhttps://www.blogger.com/profile/17718913310467614671noreply@blogger.comtag:blogger.com,1999:blog-31819641.post-67088981597261627782008-01-28T15:19:00.000-08:002008-01-28T15:19:00.000-08:00Alaisdair -In place of the part III of this idiot'...Alaisdair -<BR/><BR/>In place of the part III of this idiot's guide requested by Mr. Idiot, I'd request instead part I of the idiot's guide to noise. My request is based on your statement that it's impossible to select an independent integration limit for the m45 peaks, because the signal to noise ratio for an m45 graph is too low. Hmm! If there's too much noise to allow the scientists to set an independent integration limit for the m45 peaks, that suggests that there may be too much noise to allow for other things, like the accurate measurement of the area of the m45 peaks. <BR/><BR/>So maybe noise is an important topic.<BR/><BR/>On your main argument, you must be right that Brenna and MA came to different conclusions because they were working from a different set of assumptions. Yes, Brenna probably based his conclusion on a corrected m45 peak that was in effect pushed further to the right than MA's corrected peak. (There are other possible differences in the assumptions used by Brenna and MA -- for example, perhaps they used different assumptions about the shape of the m45 peaks.) <BR/><BR/>I'm not sure why Brenna and MA would use different assumptions to reposition the m45 peak. Isn't is pretty well known exactly how far the m44 peak lags behind the m45 peak? (it's something like 150 ms, isn't it?)<BR/><BR/>If I'm following your argument (where you say, for example, that "we are not talking about the actual lag between the peaks"), then ALL of the methods used in the EDF reprocessing involved some effort to align the m45 peak with the m44 peak. That makes sense. This means that the LNDD manual processing method required the technician to first manually align the m44 and m45 peaks, then manually set the integration limit. In contrast, the automatic method used during EDF reprocessing would have automatically aligned the m44 and m45 peaks, then automatically set the integration limit.<BR/><BR/>If this is the case, then how can we say anything about whether EITHER of these two methods (manual and automatic) is like the method used by Dr. Brenna? True, the automatic processing method produced more negative results than the manual processing method, but we have two possible factors to account for this: the m45 alignment factor and the integration limit factor. Don't we have one too many "factors" to allow us to reach any conclusions?Larryhttps://www.blogger.com/profile/08976868079076669453noreply@blogger.comtag:blogger.com,1999:blog-31819641.post-25212208242294034642008-01-28T13:08:00.000-08:002008-01-28T13:08:00.000-08:00syi,I'd be very surprised if any compensation appl...syi,<BR/><BR/>I'd be very surprised if any compensation applied is of a fixed nature. That would be an incredibly risky thing to do. Having said that, I can't say categorically that isn't done. My understanding is that it is done on the fly, each time an integration is performed.<BR/><BR/>Larry's comment regarding the m44 and m45 being separated during combustion is correct (at least that's what I thought was the case).<BR/><BR/>Having said that, I know that you know that already ! (a forehead slapping moment for you ?)<BR/><BR/>:-)<BR/><BR/>AliAlihttps://www.blogger.com/profile/00786387057717601356noreply@blogger.comtag:blogger.com,1999:blog-31819641.post-44592418548712481202008-01-28T12:17:00.000-08:002008-01-28T12:17:00.000-08:00syi -I'd like to see the quote from Dr. MA that yo...syi -<BR/><BR/>I'd like to see the quote from Dr. MA that you're referring to. In theory, the polarity of the GC column should have no effect on the alignment or misalignment of m44 and m45 peaks, since we don't have separation of m44 and m45 until after combustion of the substance in question (after the material has passed out of the GC column). But if I'm missing something and the column really DOES affect the m44 - m45 alignment ... that becomes a very powerful argument for FL. First, it helps illustrate the relevance of the column switch, and increases the potential burden on USADA to prove that the column switch did not cause the AAF. Second, it eliminates the need to resolve the battle of the experts. If the column switch caused a greater m44 - m45 misalignment, then FL can prove his case without proving that Brenna was wrong. <BR/><BR/>So, inquiring minds want to know more about this Dr. MA quote.Larryhttps://www.blogger.com/profile/08976868079076669453noreply@blogger.comtag:blogger.com,1999:blog-31819641.post-16204133347855806492008-01-28T09:21:00.000-08:002008-01-28T09:21:00.000-08:00That's great stuff, Ali. Very clear.Here is one p...That's great stuff, Ali. Very clear.<BR/><BR/>Here is one possible relevant detail. I read somewhere that Dr. MA said that the degree of misalignment (the isotope effect) depends on the polarity of the column used for separation. If LNDD used one column, but set their software for some automatic adjustment appropriate for another column with a different polarity, that could account for some of the inaccuracy.<BR/><BR/>Hmmm...could be a relatively "innocent" explanation for the whole problem. They switched columns in order to get better separation of the metabolites, but, not understanding the big picture, failed to make all the necessary adjustments to other parts of the system. Hmmm...<BR/><BR/>Also, is there a Part III? In which you show us how things completely fall apart when there is a third peak present with unknown CIR? We know that little peak is there in between the 5bA and the 5aA from the GCMS chromatograms.<BR/><BR/>syiMike Solberghttps://www.blogger.com/profile/11784753552166129987noreply@blogger.comtag:blogger.com,1999:blog-31819641.post-52579333493605204292008-01-28T08:48:00.000-08:002008-01-28T08:48:00.000-08:00Larry,OK, the main reason is that in practise, bec...Larry,<BR/><BR/>OK, the main reason is that in practise, because the m45 signal is tiny, it is subject to interference from chemical and electrical noise. This may make it difficult to identify exactly where the peak starts and stops (and we know how sensitive the results are to even tiny errors in the integration). The m44 signal, in comparison has a very good signal to noise ratio, so it's far easier to see exactly where to place your integration limits. Ergo, align the peaks using techniques such as "centroiding" and set your integration limits using the m44 peak (the short answer would have been "it's what Brenna did")<BR/><BR/>Huh ? You've been working too hard on that opus. Yes, there are separate detectors for m44 and m45 which generate two separate signals. I think you'll find I said that ? (at least I thought I said that !!!)Alihttps://www.blogger.com/profile/00786387057717601356noreply@blogger.comtag:blogger.com,1999:blog-31819641.post-1171243310687946682008-01-28T08:26:00.000-08:002008-01-28T08:26:00.000-08:00From a layman's point of view, your analysis makes...From a layman's point of view, your analysis makes a whole lot of sense.<BR/><BR/>If you guys are idiots, what's that make LNDD? Amoebas?<BR/><BR/>RSDRubber Side Downhttps://www.blogger.com/profile/11291956379296097645noreply@blogger.comtag:blogger.com,1999:blog-31819641.post-75936569020839813942008-01-28T08:23:00.000-08:002008-01-28T08:23:00.000-08:00Alasdair -Why apply the same integration limit to ...Alasdair -<BR/><BR/>Why apply the same integration limit to both the m44 and m45 peaks? Why not graph them separately and apply separate integration limits?<BR/><BR/>My understanding of the IRMS machine is that it DOES contain separate particle detectors for m44 and m45.Larryhttps://www.blogger.com/profile/08976868079076669453noreply@blogger.com