What is documented where
Following up the revelation that LNDD used different columns on the S17 when their standard operating procedure (SOP) says they should be the same, we went looking for documentation, and got some surprising results. Here is a summary of our examination.
| Date | UCI Sample | Exhibit | SOP MS | SOP IRMS | Actual MS | Actual IRMS |
| 7/3 | 993865 | Ex 92 | LNDD 1427 | LNDD 1453 | missing | missing |
| 7/11 | 994203 | ? | ? | ? | ? | ? |
| 7/13 | 994277 | Ex 88 | LNDD 1045 | LNDD 1071 | missing | missing |
| 7/14 | 994276 | Ex 90 | LNDD 1236 | LNDD 1262 | missing | missing |
| 7/18 | 994075 | Ex 86 | LNDD 853 | LNDD 879 | missing | missing |
| 7/20 | 995474A | USADA LDP | missing | USADA 153 | USADA 124 | missing |
| 7/20 | 995474B | USADA LDP | missing | USADA 329 | USADA 303 | missing |
| 7/22 | 994080 | Ex 87 | LNDD 951 | LNDD 977 | missing | missing |
| 7/23 | 994171 | Ex 84 | LNDD 664 | LNDD 690 | missing | missing |
| Aguilera | NL1 | Ex 89 | LNDD 1142 | LNDD 1168 | missing | missing |
| Aguilera | NL2 | Ex 85 | LNDD 758 | LNDD 784 | missing | missing |
| Aguilera | NL3 | Ex 93 | LNDD 1521 | LNDD 1547 | missing | missing |
Table 1: Where column types are identified
Looking at this altogether, here are a few points of note:
- All the tests specify MAN-52 and MAN-41 as the SOPs and they call for the same column, a DB-17.
- Nowhere is there an indication of what column was actually used for the IRMS, only the SOP pages. We don't know what column was used for any IRMS test from the documentation. With the data now available, it is no longer safe to make assumptions.
- The indication of column used for the MS is only present in the samples run on IsoPrime1, there are none for IsoPrime2. We don't know what column was used for the MS on the alternate B samples. We similarly have no reason to trust the SOP and execution agree.
- The SOP pages for MAN-52 should have come at the beginning of the IRMS section in the LDP, but we get the "actual" page only.
- The "actual" pages for column used should have been present in the alternate B's next to the SOPs, but were not there.
If anyone paws through the exhibits and finds things we've missed, please let us know.
90 comments:
I think you mean to say "...making it IMpossible for a qualified expert to determine ..."
I can see how it might have read that way. I clarified the wording.
thanks,
TBV
On the subject of LNDD's competence, anyone that is interested in getting a glimpse at the quality of their work should read Vrijman Report for UCI on WADA, Lab, Armstrong (it is in the Links of Interest section of TBV's main page). Reading just the Executive Summary and Conclusions is enough to get a good sense of what a schlock (and shady???) operation that lab is. The comments about WADA's violations of its own codes are also quite illuminating.
bill mc, agreed 100%. I think that Vrijman was way, way too gentle with LNDD in his report.
TBV ... if I follow you, then we don't know what column LNDD used in its IRMS testing ... and we're not sure whether they actually used a different column for the MS and the IRMS? All we know for sure is that LNDD did not follow its SOP for MS testing? And that the column they actually used for MS testing does not seem to be designed for steroid testing?
BTW and FWIW, there's a THIRD column type mentioned in the lab package. Apparently, LNDD used an Agilent 19091Z-002 to do the T/E testing. See USADA 266. I can't find any information on what column is specified in the SOP for this test, and I don't know anything about this particular column.
Larry,
Agilent 19091Z-002 is the part number for one of the HP-1 model columns (it appears to be an HP-1 with a .20mm inner diameter, 25 m height, and .11um film thickness). Like the HP-5ms below, it is non-polar, but its composition differs slightly from that of the HP-5ms. Its listed uses are for amines, hydrocarbons, pesticides, PCB's, phenols, sulfur compounds, and flavors and fragrances.
The DB-17ms, the model that the SOP's seem to require, is mid-polar and has as listed uses drugs, glycols, pesticides, and steroids.
Finally, the third model, the HP-5ms, is non-polar and is the one shown in the documentation as having been used for the two 7/20 tests in the chart above. Its listed uses are for semivolatiles, alkaloids, drugs, FAME's, halogenated compounds, herbicides, and pesticides.
All of the above is from the Agilent website. You might want to confirm that I got this right.
Lastly, from TBV's latest information it appears that the documentation shows only the use of the HP-5ms for the 7/20 tests. Other than your latest find about the HP-1, there apparently is no indication of what columns were used for any other tests, though again the HP-17ms seems to be the required one for the MS and IRMS tests.
That's my understanding of things as they stand now. Hope this helps.
I meant DB-17ms in the 2nd to last paragraph above.
rbp, thanks
So, TbV, you are saying that we don't know for sure that they used different columns, right? We only know for sure that they used the wrong column (the HP-5ms, rather than the DB-17ms) on the GCMS, but we don't know for sure what they actually used on the IRMS. They might have used the wrong one on the IRMS also. If they did that, it could be the same column for both, although the wrong column according to their SOPs. Of course, they could have used any other column sitting around in the shop as well. Have I got that right?
syi
Mike,
yeah, that's right. We don't have any documentation about what column was used in any IRMS.
We now have no reason to believe they follow the SOP, so without evidence, who knows?
TBV
A couple of loose ends here:
First, TbV, you are right that the SOP for the 7/20 GCMS is missing from the LDP. However, the effective date of the GCMS SOP (M-AN-52) on LNDD 951 is 28/10/2005 (Euro style). This is the document they provide for a test that was done in April, 2007. So we can be sure that the same SOP applied in July, 2006. Even if they didn't actually use the required column (DP17-ms) in April, 2007 (and we don't know whether they did or not), the given SOP would have still been in effect in July, 2006.
So, I think you can fill in the "missing" of the July 20 GCMS test, with "missing, but known to be SOP M-AN-52" or some such comment.
Second: I don't think we have yet named the actual ISO 17025 violation. ISO 17025 5.4.1 says:
All instructions, standards, manuals and reference data relevant to the work of the laboratory shall be kept up to date and shall be made readily available to personnel.
Deviation from test and calibration methods shall occur only if the deviation has been documented, technically justified, authorized, and accepted by the customer. emphasis added
So, given that the LDP contains no documentation of the reasons for the change to the MS-5ms (or the '433 as I think you are calling it), there is a clear ISO violation, and thus a "burden flip."
So can USADA claim that the change to the wrong column did not cause the AAF?
Well, if they also used the wrong column in the IRMS (thus using the '433 in both GCMS and IRMS) then I suppose they could claim the chromatographic conditions were the same, and the violation(s) didn't cause the AAF.
But if they did, in fact, use different columns in the GCMS and the IRMS (and, of course, we don't know for sure what they used in the IRMS), then I don't see how they could prove the violation didn't lead to the AAF. It is clear that some substances (even if not the metabolites of interest) move around between GCMS and IRMS with different columns, so they have no way to prove there is no co-elution/contamination of the peaks of interest in the IRMS, especially given the drawbacks of the IRMS 'grams.
syi
SYI,
I think you've got the first level of analysis correct. This is basically what we had at the time of the revelation.
What I'm looking for are possible missing pieces that could be sprung to mess up arguments.
Here are a few.
1. It turns out the use the 5ms/'433 on the IRMS. So they have consistent conditions, leaving us with previous identification/specificity issues.
2. They produce or manufacture an updated MAN-52 and MAN-41 effective May 2006 specifying the '433, making the ISL and ISO violation go away.
3. They produce mass-spec data for the alternate B samples that shows nothing untoward. Subcases: (a) same columns DB-17 on B's. (b) same columns '433 on B's; (c) different columns on B's.
For 3(a), this would be the most effective argument for Truth of the AAF that can be provided with the evidence that might now be available (the MS for the S17 being erased). It doesn't address the ISL violation on columns for the S17, but might be taken as proof they didn't "cause the AAF". That's why they were run, after all.
For 3(b), we are left with a possible Landaluce scenario, where the Panel needs to decide if this violation is unredeemable. Maybe with Mass spec data, it could be accepted as true anyway subject to identification RTT issues.
For 3(c), I think we are left with the the initial argument -- there's a violation, we don't have the MS, and if we had it, then it wouldn't necessarily be valid because of the column change.
A lot matters whether the column change is real, because of the burden flip that would follow.
TBV
I don't see how they can claim your 2). The SOP presented for the test(s) in April, 2007 is stamped October, 2005. If they had a different version of MAN52 in effect during the summer of 2006 (specifying the '433), and then switched back again after that (to the DB17), then the SOP used in April, 2007 would have to have an effective date after the summer of 2006. So the SOP from October, 2005 must have been in effect for July, 2006.
It's not a big point, but it does make the ISO violation more firmly settled.
Your 1) and 3) are still possible, but depend on LNDD/USADA producing additional evidence, so we'll have to wait and see what will happen, I guess.
syi
Mr. Idiot and TBV, please note that the requirement to document deviations from procedures is set forth in the ISL. TD2003LDOC requires the lab to document in its Laboratory Documentation Package (LDP) any deviation from the lab's written screening procedures, "A" sample confirmation procedures and "B" sample confirmation procedures. I don't think you'll find a clearer ISL requirement than this one.
Mr. Idiot, we do not want to rely on ISO documentation requirements, as these requirements may not be applicable under the ISL. As you probably remember, ISL Rule 7.1 states that:
"The Laboratory is not required to provide any documentation not specifically included in the Laboratory Documentation Package. Therefore, the Laboratory is not required to support an Adverse Analytical Finding by producing, either to the Testing Authority or in response to discovery requests related to the hearing, standard operating procedures, general quality management documents (e.g., ISO compliance documents) or any other documents not specifically required by Technical Document on Laboratory Documentation Packages. References in the International Standard
for Laboratories to ISO requirements are for general quality control purposes only and have no applicability to any adjudication of any specific Adverse Analytical Finding."
To be honest, I'm not sure how to read ISL Rule 7.1 consistently with ISL Rule 5.2.6.6, which DOES seem to require the Lab to produce documents required by the ISO. However, given the language I quoted above from TD2003LDOC, we probably do not need to rely on the ISO to prove an ISL departure based on LNDD's failure to document its departure from its SOP.
I have not completed my research, so I don't want to sound too confident here ... but if the LNDD actually used a GC column that's different from the one required under its SOP, and if the LNDD failed to document that they had done so, this looks like an obvious ISL departure to me.
M, if you're listening, do you agree?
Thanks Larry. That's fine with me. A TD2003LDOC violation it is then. You lawyers aren't all bad.
I do seem to remember that there is some way in which the ISO violation is important, but I can't remember the textual source of that.
Regarding TD2003LDOC, remember that it says:
The items listed below do not constitute a list of required flow charts, forms or documents, but instead a list of information necessary to support the analytical
result.
I think that means they have to produce whatever they have to produce to show the AAF is valid. Surely that would include documentation of which column was used in the IRMS.
syi
SYI, I can't speak with any authority about ISO 17025. My understanding is that the criteria under ISO 17025 are general in nature, and that ISO 17025 is supposed to be supplemented with more specific requirements for specific fields (annex B.4 of ISO 17025 is supposed to speak to this). So I guess the ISL is supposed to contain the specific requirements for doping testing, and that ISO 17025 is supposed to contain more general requirements.
My understanding here is consistent with ISL 5.1, which reads in part as follows:
"This section of the document is intended as an application as described in Annex B.4 (Guidelines for establishing applications for specific fields) of ISO/IEC 17025 for the field of Doping Control. Any aspect of testing or management not specifically
discussed in this document shall be governed by ISO/IEC 17025 and, where applicable, by ISO 9001. The application focuses on the specific parts of the processes that are critical with regard to the quality of the laboratory’s performance as a Doping Control Laboratory. These processes have been determined to be critical
to the defined ISO 17025 criteria and are therefore determined to be significant in the evaluation and accreditation process. This section introduces the specific performance standards for a Doping Control Laboratory."
So you might read ISL 5.1 as incorporating all criteria of ISO 17025 that are not specifically addressed in the ISL. However, it's not clear to me whether ISL 5.1 can be read this broadly. It may be that the ISL incorporates ISO criteria for purposes of accreditation and auditing only, and not for determining which standards can be used by an athlete to overturn a doping finding. Again, I'll point to the language of ISL Rule 7.1:
"References in the International Standard for Laboratories to ISO requirements are for general quality control purposes only and have no applicability to any adjudication of any specific Adverse Analytical Finding."
Admittedly, I'm reading this provision out of context (Rule 7.1 addresses only the documents that have to be included in the LDP), so maybe this provision needs to be read narrowly. I'm not sure. Again, I'm not the right person to speak about the ISO. My main point is that, IMHO, a departure from the ISO is not necessarily a departure from the ISL.
I'd welcome comments and corrections from anyone who knows this area.
Regarding your point on TD2003LDOC: you note that TD2003LDOC states that "The items listed below do not constitute a list of required flow charts, forms or documents, but instead a list of information necessary to support the analytical result." I disagree that this provision requires the lab to produce whatever documents are necessary to show the validity of the AAF. The provision does not state that the lab has to produce anything IN ADDITION to what is "listed below." I think instead that this provision indicates that the list is general in nature, and that it's up to each lab to figure out what forms, charts and documents are required to satisfy each criteria listed in the TD. But if a matter is not covered in the list, I don't think the TD requires the matter to be documented, even if the matter is necessary to show the validity of the AAF.
I mean, if the lab did not have to produce complete mass spectrum data, then clearly they don't need to document everything required to show a valid AAF.
So, what do you think of lawyers now? ;^)
Mike,
don't see how they can claim your 2). The SOP presented for the test(s) in April, 2007 is stamped October, 2005. If they had a different version of MAN52 in effect during the summer of 2006 (specifying the '433), and then switched back again after that (to the DB17), then the SOP used in April, 2007 would have to have an effective date after the summer of 2006. So the SOP from October, 2005 must have been in effect for July, 2006.
I meant that USADA/LNDD present new evidence in the form of updates to MAN52 and MAN42 dated before the 2006 tests, not that they have done so.
Whether such new evidence could be believed is a different matter -- I'm just saying it is the kind of surprise, like newly found mass-spec data, that could confuse things.
TBV
TBV, is there a fourth possibility? That LNDD actually used the right column on all of its tests, and that the documentation is in error?
This raises the question of whether the column is identified by the machine (in which case we can assume that the identification is correct) or whether the column is identified by the lab technician (picked from a drop-down list or something like that).
Anyone know the answer to this question?
Larry, an excellent question. The manual was lying around somewhere. If I can find it, I'll look for an answer.
I'd like to hope it was automatic, but then again I doubt the coil of tube has any electronics to provide an ID. That makes me think a drop down is very possible.
Of course, it being documented wrong is a violation itself, but it's hard to say it would affect the reliability of the test if there were another way of proving it was right.
I'd suspect the maintenance logs ought to show column installation and swaps, if they use more than one column type in the machine.
One maintenance log was produced late in the hearing, for other reasons. We don't have that in our pile of exhibits.
TBV
TBV, forgive me for pushing this, but I still think you don't get my point about your point 2). How could they possibly produce "new evidence in the form of updates to MAN52 and MAN42 dated before the 2006 tests"?
If they had updated MAN52 or MAN41 before July, 2006, to designate the '433, then they would have had to change it back to the DB17 before April, 2007. If they did that then LNDD 951 (et al) would have to have an effective date beginning sometime between August 2006 and April 2007. But it doesn't. It's October 2005.
The only way your point 2) could work is if they claim to have had two different MAN52 documents in effect at the same time, and I don't think even LNDD would claim that.
I still think you can change your chart to reflect that MAN52 was in effect for the July 20, GCMS. I think the use of the HP-5ms/'433 is a logically unavoidable violation of their own SOP. Do you disagree with that?
It seems to me that you are leaving the door open for the use of the '433 to be explained away somehow. I don't that can be, short of Larry's suggestion that they just recorded the wrong machine, and actually used the DB17. Far fetched, but possible I suppose.
syi
Ah, I see the grounds of the confusion, I hope.
The chart just says what documents are where in which package. It doesn't say what is in effect at a particular time.
My hypothetical is that they come up with a pair of documents dated May 2006 changing to the '433, and say, "oops, all the ones using the DB-17 you've gotten are innocent errors. A low paid clerical employee put the old version in by mistake."
I am fetching far about how they might try to explain away the use of the '433. I'm not saying it will work as an explanation, just imagining how it might be tried. But with arbitration, such an attempt might be as good as Brenna's "eyeballing" explanation was in Round One.
Larry asks a really good question whether we can be sure they did use the '433, or will claim that its appearance in the LDP was an error, and they really used the DB-17.
TBV
If the column ID is not automatic, but rather either a pull down, radio button, or even a basic fill-in-the-blank; what believable basis would LNDD have to claim that a different column than that IDed was actually used? Would they even know at this point?
Explaining away an apparent discrepancy well after the fact by simply saying "Oh, we actually used the right column, believe us" without some corroboration doesn't cut it IMO.
"The chart just says what documents are where in which package. It doesn't say what is in effect at a particular time."
Okay. That makes sense.
So, yes, there are a couple of far fetched ways they could claim your 2) above (10:32 post). But short of that, the use of the '433 is an TD2003LDOC violation.
Your 1) and 3) are still possible, but I don't think they can come out of this looking good in any of your scenarios.
syi
wschart, if LNDD claims that the documentation was incorrect and they really used the right column for both tests, then fairness would require LNDD to provide corroborating evidence. However, the testimony of one of the lab technicians might be enough corroboration. It might be possible as a matter of science to look at the chromatograms and say, this or that chromatogram could not have been produced by column A, it could only have been produced by column B.
The use of the wrong column in the GC-MS test is such a gross error on LNDD's part that you have to consider whether there might be some relatively innocent explanation for the documented use of the wrong column. If we had more confidence in the LNDD, we'd probably assume that the documentation was wrong: at a good lab, we'd expect to see a paperwork screw-up before we'd expect to see a procedural screw-up.
But we have another fact to play with: the lab documentation indicates that a THIRD column was used in the lab's T/E testing. Let's assume for the moment that the column shown for the T/E testing is ALSO the wrong kind of column for this kind of test (this is probably a safe assumption; my guess is that the same column is required for the T/E tests and the CIR tests). In that case, we have a second mismatch between the lab's documentation and its SOP for column selection. I think this makes it less likely that the problem was with the documentation and not the column selection.
The FL team may be in the strange position of trying to defend the accuracy of the lab's documentation, while USADA will be forced to argue that of course you can't believe the LNDD's documentation.
This is one strange case.
Perhaps this is another possible claim LNDD could make. This article:
http://www.diab.com/files/a228-401.pdf
introduces something called "Method Translation and Retention Time Locking."
This is from 1998 (so it's old) and this article only has to do with pesticides, but perhaps they have developed the same idea with steroids.
Basically, it seems they have developed the specific alterations to be made in injection pressure and oven temperature that will make substances elute at the same time no matter the column. You enter the dimensions of the column (and some other info) and the program tells you what injection pressure and oven temp to use to make the substances come out in the right order at set retention times.
It doesn't seem they make any adjustment for the different linings of a column. I can't figure out why not.
You would think we would have heard about this if it was happening, or maybe they can't do it for steroids, but who knows what they will claim.
syi
They use an HP/Agilent 6890 for the MS. I haven't yet found how it knows what columns are present -- it appears to be manually configured. According to this article:
When reinstalling the ChemStation software, it is often desirable to recover the 6890 GC column catalog inventory from the previous installation. The following is a procedure for performing this recovery.
The 6890 GC column catalog inventory is stored in four (4) files in the d:\HPCHEM\DRIVERS subdirectory.
To recover the column catalog inventory copy the following files from the previous installation to the new installation
d:\HPCHEM\DRIVERS subdirectory.
CATALOG.DB
CATALOG.PX
6890COL.DB
6890COL.PX
Where: d = drive designation where the ChemStation software is installed.
It kind of looks like you manually program in what kinds of column you have, this is kept in a catalog, and you select what column you want to use via software. It appears that LNDD has only one column installed, and their catalog has identified it as the '433.
According to the "Agilent 7890A Gas Chromatograph Advanced User Guide", on page 27, configuration consists of:
You define a capillary column by entering its length, diameter,
and film thickness. You then enter the device controlling the
pressure at the Inlet (end of the column), the device controlling
the pressure at the column Outlet, and the Thermal zone that
controls its temperature.
It doesn't mention entering the column type or model number.
While promising, the "Agilent 6890N
Gas Chromatograph Troubleshooting" doesn't seem to have much interesting, though it does say on Page 9:
Column configuration
Reconfigure the GC every time a column is trimmed or changed.
Also verify that the data system reflects the correct column
type, length, id, and film thickness. The GC relies on this
information to calculate flows. Not updating the GC after
altering a column causes incorrect flows, changed or incorrect
split ratios, retention time changes, and peak shifts.
(emphasis added).
On page 28, for bad resolution it suggests:
Set column flow to optimum linear velocity.
• Install and use deactivated consumable parts in the inlet (for
example, a liner).
• Perform column maintenance: Bake out contaminants,
remove the contaminated length of column near the inlet,
and reverse and bake out the column as needed.
• Check column installation at both ends.
• Select a higher resolution column.
For tailing peaks, on page 30 it suggests:
Which peaks are tailing?
• Are the tailing peaks active compounds, all compounds, or
are there trends (such as early eluters or late eluters)?
• Check the column for severe contamination.
• Consider the column stationary phase (active column).
• Verify that the column was cut and installed properly.
• Consider the type of adapter, liner, and inlet seal being used.
One or all of these may be contaminated or active.
• Check adapters (if installed) and liner for solid particles.
• For capillary splitless injection, consider compatibility
between the solvent and column.
• Verify that the injection technique is adequate.
• Verify the inlet temperature.
• Check for dead volume in the system. Check for correct
column installation at both ends.
• Inspect any transfer lines for cold spots.
And for FID baseline > 20pA, on page 39:
• Verify the purity of the carrier and detector gas supply.
• Inspect the column for column bleed.
• Check the gas supply trap indicators/dates and ensure that
the traps are not expended.
• Verify that the detector was reassembled properly after
recent maintenance.
• Inspect the detector for contamination.
• Check that the FID leakage current is < 2.0 pA. (See “To
Measure FID Leakage Current” .)
Raw info above -- I'm not sure what it means.
TBV
Sorry, I obviously can't sleep...
The whole GC machine used in the July 20 tests was an Agilent 6890. This document:
http://www.chem.agilent.com/Library/support/
documents/a16121.pdf
talks about how to initialize a column on the 6890. Maybe somebody can get more info from this than I can, but it seems you don't enter the specific column anywhere, just the column dimensions.
The two columns that might be involved here have the same dimensions, but different substances as "liners." Those internal liners make all the difference in elution time. But it would appear that the set up of the machine is the same for both.
I haven't figured out how the actual column is identified, but from what I have seen it is probably a drop down menu of columns that have been previously added by the lab/user.
Obviously there is more to learn here!
syi
Sorry, TbV, we overlapped. Pretty much the same info, although I think the "configuring a column on the 6890 GC" document is a little more helpful than the "reinstalling" one you used.
syi
Mike, FWIW, the 6890 GC is capable of retention time locking. See http://www.chem.agilent.com/temp/rad0880E/00000336.PDF p. 22.
It is the middle of the night and I'm reading the operating manuals for chromatography equipment. I must be losing my mind.
Larry,
I wanted to comment on the "third" column used in the T/E tests. It is not surprising that their should be a different column used in this assay, since this is a different analysis procedure--they would not automatically be consistent. In fact there are a number of important differences, particularly in the sample preparation where the derivatizing agents are of completely different type (trimethylsilyl, TMS for the T/E ratio vs. acetate for the GC/MS and IRMS). The T/E assays are not comparable to the CIR assays.
Cheers,
Duckstrap
Upon further reading I found that Retention Time Locking requires use of "the same nominal column."
One of the documents on the Agilent website says:
"RTL provides the ability to match chromatographic retention times exactly in any 6890 GC system to those in another chromatographic system with the same nominal column. The RTL software allows rapid, accurate locking of all retention times using column of the same stationary phase and dimensions (same part number)."
So that option is out for LNDD.
syi
Larry:
If there is something in the graphs that shows definitely whether it is from column A or column B, that would work. However, a lab tech's testimony: isn't that sort of self-corroboration? "The documentation is wrong, we actually used the right column. Proof? Well, I said so."
Duck, I understand your points about the "third column". The "third column" is an HP-1, which is supposed to be similar to the DB-1ms. If that's true, then you might get a kick out of this chromatogram on the Agilent web site, which (in my unscientific opinion) seems to indicate that this column cannot really separate testosterone from epitestosterone.
Steroid chromatogram.
TBV, I am continuing to research what it means for the LNDD report to show the column model. What follows is my best reading of the stuff I've been able to find. As you know, I'm not a scientist, so my conclusions here may be completely wrong. Please review my work here.
1. The LNDD GC does not automatically detect the type of GC column installed in the GC. See GC 6890 Operating Manual p. 46.
2. The LNDD lab operator does not have to configure the GC with a description of the installed column. However the GC operating manual says that such configuration is "extremely desirable" for capillary columns like the ones used at LNDD.
3. I conclude that LNDD did configure the GC with column descriptions. There is a lot of evidence to support this conclusion: (a) such configuration is recommended, (b) the data required for this configuration - column length, diameter and film thickness - are all specified in the GC configuration shown in the LDP, and (c) such configuration allows the GC to calculate flow rates and velocity, and this information is also shown in the LDP. For the information required to configure a column for this GC, see GC 6890 Operating Manual pp. 48-49.
All of this is interesting, but I still can't figure out what it means that the LDP shows the column model number. I don't see anything in the operating manual that talks about this. My guess is that the inventory catalog mentioned in your cite from last night contains a database of column lengths, diameters and film thicknesses (and probably also maximum temperatures) by model number, and that there's some way for the lab technician to describe the installed column by selecting the column model from this database (rather than having to enter this data for length, diameter and thickness). Of course, it COULD be the reverse - that when the operator manually enters the column length, diameter and thickness, then the system shows the column model number to the operator (as a QC check).
It would be nice if we could check with someone who actually works with this model of GC.
Good work Larry. I think your very last point is off. I don't think it could be the reverse, because many more than one column has the same length, diameter and thickness. In many cases the physical dimensions are the same, it's just the "stationary phase" or what I think of as the "internal liner" that differs.
"It would be nice if we could check with someone who actually works with this model of GC." Hey, that's an idea. Duck, you must know somebody.
syi
Mike, thanks for the compliment.
Yes, you'd think after all these years of working with databases, I would understand the nature of a one-to-many relationship.
OK. So we're guessing for the moment that the LNDD GC allows the technician to define the installed column by selecting the column model number from an inventory list ... and that this selection is indicated on the reports contained in LDP. If this guess is correct, then the indication of the column model number in the LDP IS significant -- it's not just a meaningless "fill in the blank", it's used by the GC to set up important chromatography operating conditions.
This doesn't mean, of course, that the column shown in the reports necessarily matches the column that was actually in use by the LNDD. But it WOULD indicate that any mis-match would be a functional error on the part of LNDD, and probably an ISL departure.
Okay, I figured it out. Larry, look at the Operating Manual p. 180 and following pages. The column is entered along with several other parameters, which are all grouped together as part of a user defined "method." You can see the details in the Operating Manual. All the items listed in USADA 124 and 125 are there.
The long and short of it is that the answer is "no." LNDD cannot claim that they just entered the wrong column in the documentation. The column is a part of the user-defined method, and the user just selects which method will be run for that test.
I haven't thought through this completely, but it seems to me that this looks even worse for LNDD. Although we don't have the proper document that says so, we know from deduction that the GCMS SOP shown on LNDD 951 (for example) was in also in effect in July, 2006. But the column listed there for method "M-AN-52" (the DB17) does not match the column actually run with method file M-AN-52. So their documentation can't be trusted. Big surprise. How many other mistakes are there like this?
syi
Cheeze whiz!
You mean we actually got something resolved?
Wow!
TBV
Where is our good friend "M"?
I declare "troll"!
The real Floyd wouldn't be silly enough to start posting sarcastic comments on a web site anymore. He writes reasoned op ed pieces now.
This has got to be a fraud.
TBV
Mike and Larry,
Sadly, I don't know anyone anymoore. Only have my own (outdated) experience with HPLC to draw on. That said, from what I can get in the manual, I suspect you need to enter the column model manually. Basically, with a little 1/4" wrench, you can install any column you want, and there is not really any way for the software to figure out what it is, except perhaps e.g. a packed metal column vs. a glass capillary column (like there is here). One place that you might want to look in the manual is under "Analytical Methods", p. 180. This allows a series of settings (temperature profiles, flows, etc.) to be saved in the instrument memory, and then pulled up and performed when needed. Note that e.g. pp USADA124-5 reference the method MAN_52.M for the GC/MS. This is likely where the column characteristics are stored. I don't see reference to a menu in the manual, and would sort of be surprised if there were one, since there are a number of vendors supplying columns, as well as the odd group who derivatize there own.
Best,
Kevin
Come on, Kevin, have you completely stopped reading my posts? It took me hours to figure it out, then you don't even read it? You must think I'm an Idiot.
syi
Any chance of a summary of exactly where we are on the subject of columns? I'm not entirely clear on what we know for a fact and what is supposition. What do we really currently know? What knowledge is required that we don't currently have?
This thing is starting to grow arms and legs so it may be worth trying to structure the investigation.
Ali
We know they were supposed to use the DB17 on both phases of the test.
We suspect they used the wrong one on the S17 MS's, because there is a document that suggests it was the '433 rather than the DB17.
But by looking at the manuals, we don't think there is anything the guarantees the model number in the report matches what was in the machine, so we don't really know.
And we're unlikely to get more illumination until there is a reply to whatever was in the appeal brief, which we assume talks about the column issue.
TBV
To break it down even more, Ali, here is how I would describe the situation wrt columns.
This all started with some information from Arnie Baker in which he stated that LNDD had violated their own SOP and used the wrong column for the GC/MS tests on the Stage 17 A and B samples, and that they therefore used different columns for the GC/MS and the GC/C/IRMS tests. Clearly a big problem, if true. We have been trying to figure out how much confidence we can have in those conclusions.
LNDD's "mode of operation" document for the GC/MS test has not been made public. However, in the April, 2007 tests of the B samples, we find a "mode of operation" document (like LNDD 951) that was in effect during July, 2006. This document shows us the official configuration and settings for a testing method called "M-AN-52." When you run a test, you are supposed to be able to select a method, previously defined by the user, and have the machine do as that method dictates. The document shows that when you are testing a GC/MS sample with the method M-AN-52, the column you are supposed to use is the Agilent DB17.
However, in the LDP from stage 17, we see another document (USADA 124, for A, and 303, for B) in which someone has entered a different column, the HP-5ms (or the '433), as the column that is actually said to be running when using M-AN-52.
The real problem arises because the GC machine itself does NOT automatically identify which column is being used in the machine for any given run, but that information is manually added and shown as part of the print out of the summary of the method used.
And in the case of the GC/MS the official "mode of operation" document for M-AN-52 says one thing, and the summary of the method M-AN-52 as intended to be run for the S17 says another. They can't both be true.
So, in short, we have no idea whether the GC/MS test was done with the DB17, which is shown on the "mode of operation" document, or if it was done with the '433, which is shown on the printout of the settings of the method used.
Now for the IRMS it's a little easier, but only because we have less documentation. We know from USADA 153 and 329 that the "mode of operation" document (which explains the details of a different method, called M-AN-41) says that the DB17 is supposed to be used for the GC part of the GC/C/IRMS.
But again, we really have no idea of what they actually used. The GC part of the GC/C/IRMS does not automatically sense which column is installed, and given the lack of clarity with the GC/MS documentation, we have no reason to confidently believe that they actually used the DB17. For the IRMS, there is no summary print out of how the M-AN-41 method is actually set up in the machine.
So, the documentation is messed up, and we don't KNOW which column they used for any of the tests.
I think this means that Arnie Baker, in making his argument, put a little too much confidence in LNDD's record keeping ability. He trusted that USADA 124/303 is accurate, but I don't think that is necessarily the case. Because the GC unit does not automatically detect which column is installed, we don't know what they used.
So they might have violated their GC/MS "mode of operation" document, but we don't know. They might have used different columns in the GC/MS and the GC/C/IRMS, but we don't know.
The only thing we do know for sure is that they have conflicting documentation.
syi
Sorry Mike,
I started the writing, got distracted, then finished it and posted without looking at what you came up with. This is either great minds thinking alike or the blind leading the blind--your choice.
Kevin
A comment and an observation prompted by Mike's of 4:35 p.m.
Comment: the headers (with column data) to USADA 0124 and USADA 0303--where we would have liked to have seen the formal SOP for M-AN-52, but don't--do not strike me as values keyboarded by the operator when setting the machine up for a test. They are letter-for-letter identical, all quantitative values are identical, etc. These don't have the look of actual input values to me. They look like boilerplate run out on the occasion of each test. Consequently, I don't think they guarantee that the '433 was actually used or that anything else in them was done that way, any more than the formal SOPs guarantee that the operators followed the SOP rigorously.
(For that matter, we can't even be sure using the '433 would be a violation without seeing the full SOP. The last page of these SOPs usually includes a revision log--see USADA 0330, which completes the M-AN-41 SOP begun on USADA 0329. It's conceivable that M-AN-52 was revised after 2005 to allow using the '433.)
Observation: Those headers beginning at USADA 0124 and USADA 0303--which, as headers, you could imagine being run out to provide set-up instructions--were actually run out AFTER the tests whose results follow them. (In the case of USADA 0124 one day later; in the case of USADA 0303, some hours after one set of tests though before a second set of tests.) Can anyone think of an explanation for that practice?
--marc
Floyd asked, "Where is our good friend "M"?
I was wondering if he (mATTHEW Barnett) is busy helping USADA prepare their brief. :)
Cal
Marc, I think you are right about that information not being actual input values at the time of the sample analysis. They were entered at some time in the past as someone was setting up information about how to run method M-AN-52. At the time of the sample analysis M-AN-52 was selected from a list of method options, and 124 is the record of how M-AN-52 was set up.
I think some, but not all (i.e. the column info), of the information in the method report must control the machine. An analytical "method" is, according to the Operating Manual:
"a collection of setpoints required to run a single sample on the 6890 Series GC. Methods make it possible to restore the instrument to a
desired setup without reentering all the setpoints. You can think of a method as a collection of completed control tables, containing information such as oven temperature programs, pressure programs, inlet temperatures, etc."
Regarding your second parenthetical paragraph, see my post of 6:53 pm above.
Regarding your final paragraph, um, CYA?
syi
Re floyd 12:41: Sorry, floyd, but we will not be distracted by sarcastic comments, not even from the 2006 Tour de France champion. We have serious work to do here. I don't suppose you could help me out by telling me whether you had more than one injection of dexamethesone or methylprednisolone during the Tour last year?
Mr. Idiot
Swim,
I was wondering if you had gotten those injection dates nailed down yet -sounds like that information is still unknown if you're making a final desperate plea to floyd. Perhaps TBV could finagle that info from Floyd's MD?.
Good luck my brother -nice research on the column question.
Dan
TBV and SYI,
Thanks for information. For the cases where they've stated that they've used a particular column and submitted that as evidence, I wouldn't have thought that they could now change their minds and claim it was effectively a documenation error. There's a precedent for the CAS panel being more rigorous than the AAA one when it comes to applying WADA rules.
However, for those cases where they just havn't said, I'm sure I can guess exactly which columns they'll claim they used ... all the right ones.
My perceptions may be wrong, but the expectation of this "scientific" lab providing an honest answer is negligable. Goodbye science and truth ... damage limitation is the only thing on the menu now.
Ali
Maybe another way to figure out if they used different columns is to look at the actual retention times. I'll look at the S17 F3 A sample 'grams, because I think that is what is most familiar to people. (GCMS = USADA 171, and IRMS = USADA 173).
In all the GCMS 'grams the internal standard elutes at very close to 10.7 minutes (which I will think of as 642 seconds from here on out, because the IRMS 'grams are all in seconds). In the IRMS 'grams the internal standard always elutes very close to 867 seconds. So, for the SI there is a 225 second delay between the GCMS and the IRMS.
I'll assume that the known changes in the temperature ramp didn't change the elution time of the SI very much.
If they had used the same column, then that 225 seconds would have to be almost entirely accounted for by the combustion part of the IRMS. That 225 seconds would be the same for all peaks (not a %).
In the GCMS 'gram the 5bA elutes at very close to 910 seconds. In the IRMS the peak LNDD identifies as the 5bA elutes at 1304 seconds. That is a difference of 394 seconds. That spread is 169 seconds (394 minus 225 for the combustion time) longer than the spread we would expect if they had used the same column.
If I understand this right, there are three possible explanations for that 169 seconds.
One, the known temperature differences really did cause that much variation in retention time. I don't know for sure, but that seems unlikely. I have seen studies where temperature changes caused difference of only a few seconds, but it does depend on the substance.
Two, they used different columns. This would mean that the retention time of the 5bA is significantly greater with the DB17 column, than with the '433 column. That does make sense based on what I have read.
Three, they identified the wrong peak in the IRMS 'gram. I know this is too easy, so it must not be right, but it is interesting to me that if you assume they used the same column, and that the time lag caused by combustion is 225 seconds, then you would expect the 5bA to elute in the IRMS at 1135 seconds (910 plus 225). And, intriguingly, there is a small, but easily noticeable (but left unidentified by LNDD) peak at 1135 seconds in the IRMS 'gram.
Assuming there is some problem with my point three (because otherwise the case would have been over long ago), we have to conclude that they used different columns, which slowed down the retention time of the (alleged) 5bA by 169 seconds, and other substances by different amounts.
Have at it friends.
syi
SYI -
You wrote "I think some, but not all (i.e. the column info), of the information in the method report must control the machine." My understanding is that the column selected in the GC's method DOES control the machine, that at a minimum the machine needs to know the column length, diameter and film thickness in order to operate properly. There may be other parameters in the setup of a method that depend on the correct identification of a column, but IMHO based on a reading of the GC operating manual, at least these three parameters must be correctly set up for the GC to function.
Your 9:10 AM post is VERRRRRRRRY interesting. We know that one or more factors must account for the inability to match up the CIR GC-MS and GC-IRMS peaks by RRT. The combustion delay standing alone does not explain it. You are suggesting that a column switch MIGHT explain it. Maybe. But you're assuming for purposes of your argument that the differences in RTs cannot be explained by the differences in the GC-MS and GC-IRMS temperature ramp-up. This might not be a good assumption. It looks to me that LNDD used a much slower temperature ramp-up for the GC-IRMS than it used for the GC-MS. If I have this right, (1) for the GC-MS, LNDD used a temperature ramp-up that went from 270 degrees C to 300 degrees C in three minutes, and (2) for the GC-IRMS, this same temperature ramp-up took about 22 minutes. If I understand the science correctly, a slower temperature ramp-up will result in longer retention times, as we see in the test results.
I'm not a scientist, as we all know. But can you justify your assumption that the gap in RTs could not have been caused solely by the combustion delay plus the difference in the temperature ramp-up?
Well, it could be the temp.
Here are the actuals for GCMS:
- The column is conditioned at 70°C for one minute.
- The temperature is then ramped up to 270°C, increasing 30°C every minute; and
- The temperature is then ramped up to 300°C, increasing 10°C every minute.
So, 9 min 40 sec to get from 70 to 300.
For the IRMS:
- The column is conditioned at 70°C for one minute;
- The temperature is then ramped up to 271°C, increasing 30°C every minute;
- The temperature is then ramped up to 281°C, increasing 0.6°C every minute;
- The temperature is then held at 281°C for three minutes;
- The temperature is then ramped up to 300°C, increasing 5°C every minute;
- It is then held at 300°C for 5 minutes.
So, it would take about 21 minutes to get from 70 to 300.
Given that the temp gets up to 270 in both machines at the same pace (in 6 min 40 sec), I would be surprised if the differences in that last thirty degrees accounted for retention time differences of 169 seconds.
But maybe. This probably requires someone with actual GCMS / IRMS experience to settle. Dailbob? Duck? Others?
syi
SYI, as long as we're asking for help here, we should ask why LNDD used different temperature ramps for the two machines. Before you brought the LNDD SOPs to my attention, I had assumed that the lab had simply been careless in its selection of temperature ramps. But it turns out that the differences highlighted in your 11:05 AM post are built into the SOPs. So these differences are by design.
Maybe it's lousy design. Maybe not. It would be nice to know.
Larry, two things:
One, for background on the temp ramp issue see the "lab director admits mistakes" thread at DPF and start at post #39.
Two, let me try those temp ramp again, since I found the originals now.
For the GCMS the right schedule is this:
- The column is conditioned at 70°C for one minute.
- The temperature is then ramped up to 270°C, increasing 30°C every minute;
- The temperature is then held at 270 for twelve minutes;
- The temperature is then ramped up to 300°C, increasing 10°C every minute.
- The temperature is then held at 300 for three minutes.
For a total time of 25 min 40 sec. That also matches the x axis of the GCMS 'gram.
And the IRMS should read:
- The column is conditioned at 70°C for one minute;
- The temperature is then ramped up to 271°C, increasing 30°C every minute;
- The temperature is then ramped up to 281°C, increasing 0.6°C every minute;
- The temperature is then held at 281°C for three minutes;
- The temperature is then ramped up to 300°C, increasing 5°C every minute;
- It is then held at 300°C for 5 minutes.
For a total time of 36 minutes 40 seconds (or so). That does NOT match the x axis of the IRMS 'gram (which goes to 2560 seconds, 42 minutes, 40 seconds).
(By the way, some lab rat at LNDD can't do maths - as the above temp ramp is said to equal 45 minutes in the documentation - it doesn't.)
Now, admittedly, the IRMS procedure is based on the "mode of operation" (SOP) document, so we don't know that that is what they actually did, but that is what they were supposed to do. In fact, since the temperature procedure does not add up to the x axis of the IRMS 'gram, I would think they did not follow that temp ramp. Hmmm...is there any other explanation for that?
For the GCMS, the the "mode of operation" document and the "method file" document match, and the x axis matches, so I think that is what they actually did.
syi
For the less motivated, I strongly encourage you to at least read the DPF "Lab Director Admits Mistakes..." thread, beginning at post 62.
In the S17 B sample SOP (LNDD 329) there is a hand written correction in the temp ramp, but things still don't add up. The total must be 2580 seconds, as shown under "conditions SM" on LNDD 329, and on the IRMS x axis, but the temp instructions don't add up. Why?
Remember that the temperature conditions directly affect the quality of peak separation, so this is an important matter.
syi
swim,
Nice work.
this whole thing is getting crazy. What's being demonstrated now is even more mistakes made by LNDD than what was originally thought.
I hope Landis' team has figured out what TBV is figured out to present to CAS.
The crazy thing is your last couple posts are easy for everyone to understand and make it easier to understand how LNDD could have screwed up.
Too bad the CAS hearing want be public.
Mike
I haven't done the math, but were I to do so, I'd assume that the compounds in question boil off in the column when it hits a certain temperature, at whatever time from the start of the ramp is involved.
Then, there is to be added the combustion time in the IRMS, and the actual elution time of the column.
While the ramp time may be 36 minutes, there may be 10 minutes of the other stuff involved to get to the 45 minute cycle claimed.
They will argue the changed ramp is to increase the separation, but why they don't use the same ramp (and column!) between machines if they are trying to compare results is one of the thing that leaves folks like WMA sputtering in disbelief.
TBV
A couple other things come to mind:
1. How come USADA's experts didn't find this screw up (if it is) before this went to trial?
2. What did Brenna say to Mclaren/Brunet to make them willing to find Landis guilty. Was Brenna smart enough to 'spin' LNDDs results to make them actually believe the lab didn't screw up enough to mess up the results?
3. Is Landis going to hire SYI, Larry, Duckstap, Judge HUE, Ali to represent him at the CAS hearing?
Mike, first, thanks for the great analysis on the temperature ramps.
Mike, something is wrong here - either with the numbers, or just as likely, with the way I'm looking at the numbers. Hear me out and tell me what you think.
I'm working from two basic assumptions:
1. All other things being equal, RTs for the GC-IRMS will be greater than RTs for the GC-MS, because the GC-IRMS includes an additional step for combustion. According to the majority opinion, this additional time is constant for all substances in the mix. I'm estimating this additional time at about 4 minutes, based on the different retention times for 5aA AC shown on USADA 348 and USADA 349. So if we subtract this 4 minutes from the RTs shown for the peaks in the GC-IRMS graphs, we have an idea when these peaks emerged from the GC in the GC-IRMS test.
2. All other things being equal, substances emerge faster from a hotter GC. If we increase the temperature ramp for a GC, we should shorten RTs.
According to the GC-IRMS shown at USADA 349, the last peak of interest has an RT of about 1670 seconds. If we subtract the 4 minutes I've estimated for combustion, this peak would have an adjusted RT of about 1430 seconds. The last peak of interest in the corresponding GC-MS has a RT of 1150 seconds. This is consistent with the hypothesis I posted earlier: the GC-MS peaks emerged more quickly because the GC-MS temperature ramp was hotter than the GC-IRMS ramp.
But the hypothesis I posted earlier was WRONG.
If the last peak of interest on the GC-MS emerged from the GC at 1150 seconds, and the last peak of interest on the GC-IRMS emerged from the GC at 1430 seconds, then we need not look at the entire GC temperature ramp. We only need to look at the portion of the temperature ramp up until the RT of the last peak. And over this period, the GC-IRMS actually has a slightly hotter and faster ramp than the GC-MS. Remember, we're seeing an RT of 1150 seconds for the last peak on the GC-MS. The GC temperature at that point is about 271.6 degress. In contrast, the GC temperature at the 1150 second point for the GC-IRMS is about 274.6 degrees. By the time the final peak emerges from the GC in the GC-IRMS test, the GC temperature is about 280 degrees.
These are rough and dirty calculations, maybe someone can refine them. I suspect that the difference in the two temperature ramps is probably not all that great, but from this difference, we would expect to see RTs for the GC-IRMS (adjusted for the combusion delay) that are shorter than the RTs we see for the GC-MS. But we see just the opposite. Why?
You might have been right all along, Swim. Maybe there was a difference in columns.
(I'm reading the stuff you cited over at DPF.)
Geez,
You guys seem to be rehashing what I talked about before. The combustion phase and lower temperature ramp meant the RRTs wouldn't match up.
The temperature ramp was slightly lower to get greater separation of the 5A and 5B which are only 30 seconds apart in the GCMS. I didn't talk about how the 5A was stretched out from the IS in the GC-IRMS versus the GCMS, but that the time separation between the 5A and 5B really only increased a little, because I didn't think it was that important. However, those facts were reflected in my text graphs.
The proof is in the pudding. The chromatograms don't show any switching of the main peaks and the pattern is consistent between the GCMS and GC-IRMS. Whether or not the lab used different columns, and whether or not Landis could possibly get off on a technicality, the different columns did not cause any mis-identification. Scientifically the evidence, including the other B samples, is pretty overwhelming that Landis doped.
Probable dumb question...
I think a lot of the logic about the probability of peak switching, mis-identified peaks, and co-eluted peaks surrounding the GCMS and GC-IRMS graphs depends on the implicit assumption that LNDD performed each test ONLY ONE TIME. Is there any evidence that LNDD ran one or both tests MULTIPLE times under different conditions and then chose to keep only those results that showed an evidence of doping? The most recent posts seem to indicate that by varying temperature ramps and columns you have some control of the look and feel of the resulting graphs.
m,
I have to admit to feeling distinctly underwhelmed by the evidence. The only thing it proves to me is that LNDD should have their WADA lab status removed.
To put this in context (and leaving aside the fact that their results are just wrong), would you trust a lawyer who displayed such disregard for accurate record keeping? That's a lawyer who already has a poor reputation, by the way. I'm pretty sure I wouldn't.
Ali
To TBV 7:32 -
I haven't done the math, but were I to do so, I'd assume that the compounds in question boil off in the column when it hits a certain temperature, at whatever time from the start of the ramp is involved.
Huh? Everything is vaporized right when it is injected. It just comes off the column based on how long it takes to get through. I know you know that, so what are you saying?
The combustion chamber can't take more than 225 seconds (the difference in the SI for the GCMS and IRMS - it could be less than 225, but not more), so we still have over 155 seconds unaccounted for. That must be covered by some other component of the GC/C/IRMS machine, I'm just curious what it is. Does the post-combustion, pre-mass spectrometer component take any time? I wouldn't think so.
syi
m, we both know I am not a scientist. But the temperature ramp on the GC-IRMS appears to have been hotter than the GC-MS ramp over the relevant retention time period. This argues for peaks being more compressed on the GC-IRMS, but we're seeing peaks being more spread out on the GC-IRMS. Why? The temperature ramp does not explain this.
You should not dismiss so quickly that the temperature ramp could be an issue in this case. I quote:
"The GC column used for the two machines, I believe, was the same, including ramp conditions. If it wasn't, then it simply wouldn't be allowable to use GC/MS peaks to identify GC/IRMS peaks. You have to assume that the fractions go through the identical separation processes, or there would be no point at all in comparing the two profiles."
Who am I quoting? Not Duck. Not the good Doktor. This is a quote from OMJ!
Bill Neuman...
What you suggest, multiple injections and only showing the results they liked, is one of the possibilities raised by the log file gaps.
There's no way to know.
Similarly, there's no explanation on record why the alternate B's were completely consumed in their testing.
TBV
Bill N, I don't think that's a dumb comment at all. I think the FL team suspects the same thing. If you read through the testimony, you'll see how carefully Suh cross-examines the lab technicians on how long it took to perform each step in the testing process, and on all unexplained time gaps. I think Doktor M-A also refers to these time gaps in his testimony. I have not done the analysis, but yes, there may have been time to re-do the tests.
To be fair (and to my knowledge), the only relevant evidence here IS the time gaps. There's no evidence that anything nefarious occurred during these time gaps.
Mike ... this case gets stranger and stranger.
Yes, I think you've asked the right question, could the longer RTs for the GC-IRMS test (as compared to the RTs for the GC-IRMS test) be explained by the use of a different column for these tests? I'm no expert, but what other explanation could there be? The combustion phase accounts for about 228 seconds of the difference, but as you've stated, that still leaves a large time gap. The temperature ramp for the GC-IRMS is slightly faster than the one for the GC-MS, so that's no explanation. Could it take longer to perform ionization in a GC-IRMS? I wouldn't think so, I'd think this step would be pretty close to instantaneous. Could there be a longer time gap between ionization and particle detection? Again, I don't think so. I'm pretty sure you'd design these machines so that particles are detected shortly after they are ionized.
I'm no expert, but it appears that substances emerged from the GC in the GC-IRMS test a lot later than they emerged in the GC-MS test. I think it has to be a difference in the column used in the two tests. Again, I stress that I'm no expert, but I invite anyone who IS an expert to propose an alternate explanation.
But Mike, TBV and the rest of y'all, there's another point here, which should not get lost in our speculation about GC columns. As Mike has outlined for us, the GC-MS and GC-IRMS temperature ramps WERE different. Why were they different?
Reading through OMJ's comments over at "Lab Director Admits Mistakes" ... OMJ first assumes that LNDD must have used the same temperature ramp for both the GC-MS and the GC-IRMS. Then, once he realizes that LNDD used different temperature ramps for these two tests, he assumes (as I had) that the temperature ramp for the GC-IRMS must have been designed to increase the resolution of this test. After all, this is the KEY test, this is the test that makes the measurement of whether an athlete doped or didn't dope. You can imagine a lab saying, let's set up this test to be as sensitive as possible, let's increase peak separation for this test to the maximum extent possible.
OMJ then says (sensibly) that the higher resolution ramp should have been used for both tests.
What never occurred to me (and apparently, not to OMJ either) was that the tests as designed provide LESS resolution, LESS peak separation, for the GC-IRMS.
Why in the world would the lab want less separation when measuring the GC-IRMS?
FWIW,
This is a guess, but I suspect the reason the metabolites are retained longer IRMS, in spite of the higher temperature has to do with the differences in the columns. That is that the DB17 is a more polar column, and the acetates on the metabolites interact more strongly with the stationary phase than they would on the non-polar DB5 column, and are thus retained for a longer time. Would this switch cause the peaks of interest to switch places? Dunno. But I would guess that the more polar the molecule, the more it would be affected by the change in columns.
As an aside, based on his statements, regarding the IRMS and GC/MS requirements and the lack of MS data, I would think the OMJ should be almost forced to cede that, though he thinks it is likely that Floyd may have been doping, the testing is so bad as to be really inconclusive.
Cheers,
Kevin
Geez, I must have Aspergers...
In his second round of testimony with Suh (what's that called - re-cross?), Brenna says the following:
"It is not the case that one would see identical retention times or relative retention times
between GC/MS and GC combustion IRMS. I think you are asking me whether the interface between
the GC and the IRMS could cause a four-minute delay, and the answer is no. It wouldn't cause a four-minute delay. If everything was matched, it wouldn't cause a four-minute delay."
So, according to Brenna, the time from the end of the column to the actual detection of the ions would not take four minutes - and that INCLUDES the combustion time.
So now we have at least those 155 seconds unaccounted for and no parts of the system left to explain it. Boy, I'd like to understand this. Why Suh didn't press him on how to explain the time difference, I don't know.
Bill Neuman: Possible, but my view is that that kind of intentional manipulation of data and tests is the least likely scenario. I think LNDD is much more likely to have "just" screwed up. The tests are difficult and my impression is that the lab workers have an inadequate understanding of the big picture, which makes them unable to decide properly when variations from the letter of the law matter and when they don't.
m 11:27 - I've accounted for the combustion time and the lower temperature ramp (and everything else) and we still have at least 155 seconds of unexplained time in the IRMS 'gram. Any ideas?
Re your second paragraph, that has to do with Larry's post of 9:03 above, and I haven't digested that yet. I think he's on to something important there.
Re your third paragraph:
The proof is in the pudding. The chromatograms don't show any switching of the main peaks and the pattern is consistent between the GCMS and GC-IRMS. Whether or not the lab used different columns, and whether or not Landis could possibly get off on a technicality, the different columns did not cause any mis-identification. Scientifically the evidence, including the other B samples, is pretty overwhelming that Landis doped.
I think it is obvious that this is the only argument "your side" (although I don't mean to create an "us vs them" mentality - but the phrase is helpful) can make. Two things about that, which pretty much repeat what TBV has said before:
First, the "eyeballing" method you describe, and on which the majority decided against Landis, is not a scientifically quantifiable method, and has nothing to do with TD2003IDCR. Think of it this way, what if the patterns didn't match as well as you think they do? What if one peak was of significantly the wrong size or in the wrong place? On what basis would you decide if it was "close enough" and so find someone guilty? You have to have some quantification at some point. I think making this process about science and quantification and not about "eyeballing" is important. And the science requires certain standards that are not met in this case. The majority relied on a novel and non-scientific method to match the GCMS peaks to the IRMS, and it just isn't good enough for me, on the level of Truth and legality.
Second, even if I did accept the eyeballing, and grant that the main peaks are what LNDD says they are, you still have not answered the specificity issue. Given the different temperature ramps (not to mention the still unresolved possibility of different columns), there is no way to know that other substances are not part of those peaks. It is a real possibility, not a hypothetical. And those other substances could have skewed the CIR more negative. And then there are the integration issues that TbV and Ali have documented so well. There are just too many holes in their data and their process for me to trust that they came to the right conclusion.
syi
So basically DMA's comment about 'devine intervention' was pretty accurate.
Anyone know if he'll be a witness at CAS. I'm sure SUH, armed with this new information could blow LNDD's case wide open.
"Mike Solberg said...
Geez, I must have Aspergers..."
SYI, easy with the "Aspergers". I don't expect (nor do I want) this forum to be fully PC, but that hits close to home. FWIW.
As a follow up to my comments about the possibility of multiple tests to develop desired graphs/peaks during the time gaps. Couldn't LNDD be doing this rather innocently (from their perspective) in their attempt to "confirm" their "A" tests or to get their GC-IRMS peaks to "line up" with the GC-MS peaks--as they know they should. When they finally get one that looks good they stop testing. Essentially, multiple tests because they think the equipment is flaky, not because of malicious intent. Isn't this line of reasoning consistent with WADA's consistent attempts to test and re-test to get confirmation of positives?
Sorry. I am an Idiot, after all.
syi
I would think that information that multiple tests were performed would be material and that the results of all of the tests should be made part of the record, especially if the majority of the test results were either negative or inconclusive. LNDD, or any WADA lab, should not be allowed to "cherry pick" the results that support their conclusions while suppressing those that do not. Unfortunately, LNDD's documentation is so shoddy that it seems impossible to determine whether or not multiple tests were performed even though the testing time gaps and the consumption of the samples hint at that possibility.
Bills Mc and Neumann,
The possibility that there were multiple tests run, and the ones we see the result of "cherry picking" is quite evident. This is the whole foundation explored through the log file gaps and deletion of data complaints.
It's plausible, to me, that with a different burden of proof, that these gaps alone would be evidence of sufficient mis-feasance to leave doubt about the reliability of the results. Certainly Suh thought so -- except it doesn't seem that he fully grasped the Kafkaesque machine that was processing his arguments.
I use "mis-feasance" above instead of "malfeasance" because I agree with Bill Mc's analysis that there probably was no particularly evil-intent, just a desire to get results that match expectations in one way or another, and are consistent with what was obtained before. Whether or not they are, in fact, correct. I agree they don't understand the big picture enough to understand the flaws of the methodology that can lead to "incorrect" results actually being correct, and so could easily delete and re-run tests with "wrong" results until they get "right" ones.
TBV
Doesn't need to be Asperger's, could just be OCD. There's plenty of that running around...
TBV
Larry, that post of 9:03 was great work. I think you have it just right. The IRMS got hotter a little faster, quite the opposite of what should have happened if they were using a different temp ramp to get better separation.
It seems to me that based on all this information we have now pretty well shown that they used different columns for the GC/MS (the '433) and the GC/C/IRMS (the DB17). I don't think there is any other explanation for why the retention times on the IRMS are greater than we would otherwise expect them to be. If isothermal, stuff should elute from the DB17 slower than from the '433, so different columns is the only thing that explains the numbers we see.
So on the GC/MS they didn't use the column that is listed on the "Mode of Operation" document that was in effect in July, 2006 (the DB17). They did use the one shown on the "method file" report (the '433).
On the GC/C/IRMS they must have used the one that is shown on the "Mode of Operation" document (the DB17).
Of course, we don't know whether that was just a mistake, or if they intended to use different columns, with different temp schedules. From what I have read (and from what even OMJ has said) that makes no sense at all, and makes any comparison between the GC/MS and GC/C/IRMS highly dubious, but who knows what (or if) they were thinking.
Duckstrap wrote:
As an aside, based on his statements, regarding the IRMS and GC/MS requirements and the lack of MS data, I would think the OMJ should be almost forced to cede that, though he thinks it is likely that Floyd may have been doping, the testing is so bad as to be really inconclusive.
I thought exactly the same thing. I pressed him on that in the "science wars" thread, but he wouldn't respond.
Couple the comments Larry quoted above from OMJ, and his comment back in the Spring (March? May?) that he would "cry foul as loud as anybody" if LNDD didn't produce the complete mass spec data, I don't understand his trust in LNDD/USADA's science.
syi
I think what we are seeing is that many people involved in the Floyd Landis case, when faced with choosing between their hearts (Floyd is a doper) and their charts, which really do not support that conclusion to any reasonable degree of certainty, choose to go with their hearts and then rationalize their stance with some (meta) scientific and linguistic gymnastics.
Now, as to whether that is "mis-feasance" or "malfeasance" is a whole discussion in and of itself. Given that the anti-drugging establishment (WADA, USADA, LNDD, et.al) are really in a prosecutorial role, which brings with it a duty to find the truth, I believe the standards of conduct that they should be held to should be set quite high. Certainly much higher than what we have seen coming from them throughout this process.
Mike, I prefer to move more deliberately here. Yes, we can't explain the GC-IRMS RTs on the basis of the IRMS combustion (at least not as it was explained in the arbitration) or by the difference in the temperature ramps. A column switch is a possible explanation, and at the moment it is our only explanation.
But something troubles me. Why did LNDD design the IRMS portion of the CIR test with such a long temperature ramp? As you've pointed out, the temperature ramp for the IRMS is 11 minutes longer than the ramp for the MS. But the only time difference we know about for these two tests is for combustion, and combustion took less than 4 minutes for the S17 tests. That still leaves 7 extra minutes in the IRMS temperature ramp. What is this extra time for?
I'll throw out another thing I'm thinking about. Notice how the two temperature ramps both contain, towards the end of the ramp, a rapid rise of temperature up to 300 degrees, followed by a hold period at 300 degrees? My guess is that this rapid rise is supposed to take place after the emergence from the GC of the last peak of interest in the test. At that point, all you really want to do is to get any remaining stuff out of the GC as quickly as you can, so you can shut down the GC and move to another test.
My guess on this part of the design of a temperature ramp is supported by the MS chromatogram on USADA 348 -- the 5bP peak emerges at about the 19 minute point, just about at the end of the 12 minute 270 degree holding period. So, my guess seems to hold true for the GC-MS.
But my guess doesn't work for the GC-IRMS. If my theory is correct, then we'd expect to see the last GC-IRMS peak have an RT of around 1870 seconds (about 1645 seconds to the end of the 281 degree holding period for the temperature ramp, plus the 225 seconds for combustion). But the 5bP peak has an IRMS RT that is much earlier than this - maybe about 200 seconds earlier.
OK, granted, I'm engaging in some serious SWAGuessing here. But it IS strange, isn't it? LNDD runs its IRMS tests to the 40 minute point on the chromatogram. And for the last 12 minutes of this test, there's nothing going on at all. This is 30% of the time of the test run -- why waste all this time? Another way of looking at this: by the time you reach the 281 degree holding period for the IRMS temperature ramp, all of the substances of interest have already emerged from the GC. Why hold at 281 degrees if the GC has already done its job? Why not go straight to 300 degrees at this point, like with the MS temperature ramp?
I think we are still missing something.
Mike, Larry, TBV
I admit that I haven't been following the discussion that closely, but I have been waiting for you all to put the specificity argument into it's final form before responding. I haven't started researching that yet. But so far, Mike you in particular, have not made the legal case, that LNND was required to produce the Mass Spectra and to exclude all possibility of any other substance co-eluting at the same point as the 5A and 5B. (In fact I'm not even certain that the mass spectra could show this.)
From what you have been saying the specificity requirement is only as to validating their methods in general, and not as to the F3 sample results or any other specific testing result. Your quoting of Ayotte's testimony on cross was not at all persuasive as an admission to the contrary. So far I have not seen the legal case made that LNND must meet any specificity requirement wrt to the F3 test results, although I may have missed something from Larry in the comments.
As to the scientific possibility that some other substance could have eluted at exactly the same time as the 5A or 5b and be hidden under those peaks, and have biased the results, I don't think the discussion has moved much beyond the original speculation posed by OMJ on the daily peloton forum, and his view that the burden was on Landis to show that this had in fact occurred. Changed columns, different temperature ramps, cortisone metabolites, how has any of this been shown to have increased the likelihood of a substance hidden under the 5A and 5B peaks? Is there some peak in the F3 GCMS that has disappeared in the IRMS other than small intervening peak in the Shackleton F3? Is there some substance in Landis's urine that we know to have the same RT as the 5A and 5B?
M, I have not put my legal argument together on specificity. Moreover, I'm not sure WHAT I think the ISL requires on specificity. It is going to be a complicated legal argument, and I've been working on it for a few weeks in my copious spare time. Honestly, I don't know how it's going to come out. I understand the argument that the specificity requirement may only be applicable to validation of methods. I can promise you that I'm giving that argument a great deal of consideration.
As for the other matters we're discussing here ... yes, I get the fact that you don't see the relevance of these other matters. I see considerable relevance in the chromatographic conditions for the relevant tests, including matters such as column switches and temperature ramps. I think that if LNDD departed in any material way from its SOP on chromatographic conditions, that departure is an ISL departure ... but I'm not prepared to argue this point with you yet. TBH, I need a better grasp of the facts.
I have a gut feeling that there's something really wrong with these facts, and in our discussions so far concerning column switches and temperature ramps, we're only scratching at the surface.
I could be wrong.
For the moment, I'm in fact-gathering mode. I'd welcome your help, if you're inclined to provide it. However, given the fact that you don't seem to find any of these facts to be relevant to the case, I understand that you'll probably want to wait until I'm back in lawyer mode.
Playing devil's advocate, I had some thoughts:
1) Looking at stage 17 F3, the entire period is not taken up by the sample analysis. It is preceded and proceded by the standard pulses for quality control. The transit time of the sample, from start to stop isn't 2500 seconds. I'm not sure when they start noting the temperature profile ... after injection of the sample ?
2) Do we know the times on the graph are referenced to detection time, or is this compensated for so that it reflects exit time from the column?
3) IRMS GC transit times may be deliberately longer because more separation is required (prior to combustion and IRMS) to compensate for a spreading effect through that process?
Ali
m,
You don't include all that has transpired during these discussions. I could point to at least one cast-iron "legal" infringement which has occurred. I'm sure you know what it is.
Ali
" So far I have not seen the legal case made that LNND must meet any specificity requirement wrt to the F3 test results,"
Oh, there you are M.
Upon reflection, I think I'll withdraw my last remark. Just add "time waster" to my growing list of attributes (now "idiot", "ass", "time waster", ...)
Ali
Ali, so long as you're in self-denigration mode, I'd like to compliment you on your recent posts over at DPF. You're making a good point about the substantive meaning of violation of so-called "technical" rules. I may call on you for help the next time I have to make a "technical" argument over here.
Larry, or anyone else, I've been thinking about the different temperature ramps, and wondering if there could be any explanation for why they changed them. It seems the only reason that makes sense is that they were trying to get better peak resolution/separation. But as you (Larry) pointed out, it looks like they misjudged it. If they wanted to get separation on the key peaks, 5bA and 5aA, and whatever is near them, they didn't start the slow ramp up early enough. That is, the key substances were already gone by the time the slow ramp could help much.
What if the IRMS temp ramp was actually designed for the other column, the '433. Stuff would elute more quickly with that column and perhaps the slow temp ramp would have been perfectly placed for maximal benefit.
That doesn't make sense of why the document names a DB17 rather than a '433, but it could explain the odd use of the "too late" temp ramp.
What do you think?
syi
Mike, I've been thinking about the temperature ramps as well, along the line of "there must be a logical explanation for this." Only I can't find a logical explanation.
My understanding is that GC temperature ramps are designed to balance the need for accurate results and short testing times. The hotter the ramp, the shorter the testing time; the cooler the ramp, the greater the potential separation between peaks. There's probably more to the design of a temperature ramp than this, but I think these are good general rules to go by, based on what I've read.
Based on the above, the first explanation for different temperature ramps is the one you suggested, namely that they were trying to get better peak separation. But if ramp "X" achieves better separation than ramp "Y", then why not use ramp "X" for both the GC-MS and GC-IRMS? This is what OMJ asked over at DPF. Also, if they were trying to improve peak separation, you'd assume that they were trying to achieve improved separation on the IRMS (that's what OMJ assumed). However, since the IRMS temperature ramp is hotter than the GC-MS temperature ramp, this is not a good explanation -- based on what we know about temperature ramps, the IRMS ramp would achieve less separation than the GC-MS ramp.
So, I move to your second suggestion, which is that the IRMS temperature ramp is different from the GC-MS temperature ramp, because the IRMS column is different from the GC-MS column. You can throw into the mix the possibility that the GC pressure was also different for these two tests. Perhaps there's some very smart person at work here, who's managed to tune all of these chromatographic differences, so that they all work out the same at the end. But why do this? Why not just pick the same settings for column, temperature ramp and pressure for both tests? That's what all the scientists seem to expect.
My guess, Swim, is that there's much more to IRMS testing than I presently understand ... and that MAYBE the IRMS legitimately requires different chromatographic conditions than the GC-MS. I stress that I've never heard anyone say this, and that our best understanding at this point is that the GC-MS and GC-IRMS chromatograic conditions should be set up identically. However, if you're looking for an explanation for why LNDD set up different chromatographic conditions for the IRMS, my guess is that the explanation lies in something about the IRMS that we don't understand.
And there's a lot about the IRMS that we may not understand. For example: we've been assuming that the combustion phase of the IRMS testing adds a constant amount of time (around 230 seconds) to the RT of every IRMS peak, as compared to the RT for the same peak in the GC-MS. We take that assumption from the majority arbitration decision. But why would it be that combusion adds the same RT to every peak? I mean, how does this combustion work? With IRMS, there are ionized gas molecules that periodically enter the combustion oven, where they are "cooked" to the point where they become ionized CO2 and water, and the CO2 is released to the IRMS particle detector. How does the oven know when this "cooking" is finished? Maybe the oven just cooks every ion for 230 seconds. But there are ion peaks continually streaming into the oven, certainly at a faster rate than one peak every 230 seconds. So there are new "uncooked" peaks entering into the oven while there are still other peaks in the oven in various stages of being cooked. How does the oven know which peaks are cooked and which are not? My guess is that the oven is like a second column that preserves the order of ions as they enter the oven. But if so, isn't there the possibility that the IRMS column would change the separation of peaks from that achieved in the GC column?
These may be very, very stupid questions. But my guess here is, if there is something to the selection of LNDD's chromatographic conditions that we don't understand, it's because we don't fully understand the IRMS.
Post a Comment