Tuesday, May 22, 2007

Hearing - Tue: Today's Davis, digested

Now I know how reporters feel towards the end of an Olympic games, contemplating writing their wrap-ups and retrospective pieces, while there are still events being played out.

Today we had a complete waste of time in USADA's cross-examination of Landis, which went essentially nowhere, in desultory fashion, in front of lots of cameras. Depending on whose account you read, Landis was either tense or surprisingly relaxed. It doesn't appear to us there's anything of particular use to USADA in it, though we'll see if there's mud thrown in closing arguments. Dance, monkey, dance.



We are in the middle of something far more germane to the case, being the direct examination of Simon Davis, Landis' expert on mass-spectrometry.

Davis is the kind of scary-good expert Landis had in Goldberger for the TE and Herr Doktor Professor Wolfram Meier-Augenstein for the IRMS. He's confident and able to explain what he's talking about clearly and able to answer questions without evasions, and he wants data, not "close enough"

He helped develop one version of the Isoprime used at LNDD; he's the founder of another Mass spec instrument company and helps design the instruments. Nuts and bolts reality. And he was present at LNDD for both the retesting of alternate B samples and reprocessing of data associated with the S17 tests at issue in the case.

He started out by explaining the theory of carbon-isotope-ratio analysis, and explained sample flow through an IRMS spectrometer -- the latter being something no one had done before. He was able to explain the retangular peaks in some spectrograms as being jets of reference gas -- something no one had done before. He explained the burn-off of volatile solvents into the "FID" as a function of a valve, which explains the flat "front-porch" of most IRMS spectrograms -- which no one had done before.

When Brunet asked about the difference between a pinial and a penning gauge reading, he was able and cheerfully willing to explain the multiple stage pumps and turbines and how one gauge read one stage, and the other the pressure at another location. When the USADA brief talked about an indicator light that showed "pressure ok", Davis showed a picture of it being a pilot light in inside of the machine, reflecting the first stage pump operation, and not the actual pressure at all.

But we're getting ahead of ourselves.

We've been waiting all hearing for his appearance with the kind of details we're getting. We know the reported results look kind of funny (see Amory), but we haven't known why. Davis is the guy who is telling us, and so far he's given parts of the story, but not quite yet a conclusion.



After various preparatories, he explained what goes on in analysis. Given the raw data, one identifies peaks, subtracts extraneous background, does integration, and computation to produce the isotope ratio. The raw data is the "zero" stuff that made no sense to Brenna during the reprocessing. (We expect this thread to come back later, but it isn't yet explained.) Peak identification and background subtraction can be done either automatically, or manually adjusted.

Davis has methodological problems with manually adjusted values, because it is "without process" to make it repeatable. There are as many different processes as there are analysts, and analyst moods. While a good analyst can sometimes improve quality, it's impossible to properly resolve inadequately separated materials -- hence Suh's repeated insistence on "good chromatography".

Then Davis noted that the three operators at the lab didn't seem to know their software very well. In moving files form one machine to the other, it became apparent they'd never done it before, as he had to show them how to do certain things. They expressed appreciation at being able to do it, because now they could use the Masslynx software on samples obtained on the earlier isoprime. The Masslynx is said to be easier in some ways to use.

While Mongongu and Frelat did analysis, he observed them doing many manual steps, while taking notes about what they were doing, and asking questions. This had been annoying to Mongongu, who complained about his asking why she was doing certain things. He observed them moving peak starts and stops, and adding and removing data points to the background trace. When asked what they were doing, they'd reply, "using my experience", which doesn't seem like a repeated methodology.

All the while doing these manual adjustments, the operators were seeing the resultant CIR values. It is obvious, but so far unstated, that in doing so it is trivial to adjust starts, stops and the background to obtain a value that "seems correct"

At this point, Davis illuminated a point that had escaped USADA and Dr. Brenna: the software had a "save method" function by which the starts/stops and background selections made manually can be saved. When loaded with the sample data, one can repeatedly analyse the samples and get identical results. This was news to the staff. Their manual reprocessing had started from scratch each time, not from decisions saved from the original analysis.

So, while USADA, LNDD, and Brenna seemed to be fat dumb and happy getting numbers "close" to those of the original analysis (except when they weren't), Davis showed they should have been able to get identical numbers with no effort -- if they knew the software and were interested in repeatable results.



Then Davis skipped to tales of his observations while present during the retesting of B samples from other stages. Asked point blank if he thought the results were reliable and accurate, he said they were totally unreliable, and started into reasons.

He began here with a discussion of linearity. Linearity is the ability of the instrument to make proper isotope measurements for large peaks as well as small ones. It is typically measured by injecting a reference gas with known composition into the machine at varying pressures, resulting in varying size peaks. They should analyze to the same carbon ratio no matter the pressure. If the machine is operating correctly, it should give good results over the machine's specified range of pressures -- not the expected range of the samples, but the machine range. If it's not linear in the design range, he says the machine is broken. He pulled up a copy of the Isoprime manual showing the procedure and design range. Asked if he was familiar with the manual, he said he'd written parts of it.

We learned that LNDD does not have a copy of the Isoprime manual. We've heard the machine was delivered with a manual for an earlier, similar machine, the Isochrome instead. But Davis wasn't allowed to see it when he was there.

[It's not clear to me if there are non-linearity corrections built into any of the software; I'd kind of expect that the results of a linearity run could be saved to do corrections, but I have no idea if is done.]

Davis noted the Isoprime was a somewhat notoriously non-linear machine, for which frequent linearity checks ought to be run. He was surprised to find LNDD reluctant to do it as often as he had requested. when he asked Dr. Aquilera if he was being reasonable, Dr. Aquilera said "no comment", whatever that meant.

In any event, the LNDD SOP requires linearity runs only once a month. But, Davis noted, they may not do it that often. Buisson, the supervisor, admitted to some linearity problems, resulting in their testing the instrument at restricted ranges, presumably because they couldn't get it to work right full range.

Then he pulled up the SOP, which documented "once a month" or more frequent linearity runs. Pulling out other documents, he showed that linearity appeared to have been done less frequently, sometimes seeming to skip a month.

This SOP mistake is an ISL violation that could flip the burden of proof; it may also affect analytical accuracy.

Davis is of the belief this Isoprime drifts in and out of linearity, and that the result is changes in reported isotope ratio that depend on peak height, not material composition. That is wrong.



Next, Davis had a look at the "isoprime2", the new machine used on the retesting. He produced a picture from his cell phone, showing "Micky Mouse Ears" on the machine. These are "lifting rings", on the critical high precision magnet that are only supposed to be there during installation and maintenance on the magnet. They are not supposed to be there during operation. When present, the effect is not unlike putting a speaker near a color-TV screen, where the picture gets distorted. In the machine, the magnetic field that is machined to 0.01mm tolerance may be deflected, leading to unpredictable results. (This might account for some of the machine instability).


Then, Suh had him address an assertion in the USADA brief about a pressure measurement. The brief made it appear like an "idiot light" on the dashboard of a car. Davis showed a picture of the light in question being inside the instrument, not on the operator panel, suggesting the person who wrote the response had "a complete lack of understanding of the instrument."



Davis was asked about the importance of having and using procedures documented in the manual. Davis said this was essential, because the usual operators are not expert on the machines, and must rely on the decades of experience distilled in the wisdom of the manuals. In particular, advice on operating pressures are very important.

We were shown pages from the reports of the B samples where the pressures were outside the required range of the machine. Davis said the value suggested undiagnosed leaks, leading to contamination and incorrect results.

Brunet asked questions about various guages, and Davis happily went into geek talk about the pumps and what the gauges were measuring when.

Tellingly, Davis said the screen shots of the various samples showed the pressure all over the place, which should not be the case; the value ought to be stable, he said.

There are many effects of wrong pressure, quite possibly bad linearity. Bad linearity and low pressure seem to go together at the LNDD.


Suh asked Davis about the importance of automatic sequencing. Davis started explaining it as a way of reducing operator error. A robot produces reproducible results. By way of example, they pulled up USADA 155, the sequence chart of one of the A sample tests. As well, they pulled out the result pages in the LDP from each stage of the sequence -- USADA 157, 163, 166, 169, 172, 177, 178, 179, 180, 181 and 183.

The way it's supposed to work is that you put all the vials in the autosampler, and press start; you come back and get your summary results and the individual pages. If the run doesn't complete, you don't get a summary. It if stops, its because the operator stopped it for some reason, and restarted.

This particular series of pages, presented in the LDP as the processing of a Landis sample, was determined by Davis to have pages from other runs. The summary sheet has a "batch name", here "23016"; some of the sample sheets match; others do not, and have blanks. Further, the reported analytic numbers on the sample sheets do not match those on the summary sheets.

The same is true of results on the B sample - sheets that do not appear to match the batch, and have incorrect reported analytic values.

This was left somewhat hanging -- we don't know what it means, but there is speculation it relates to undocumented "re-injections". There is nothing wrong with re-injections, Davis says, if they are documented. But none of these seem to be documented, so we don't know what is going on.



Davis next attempted to do a live demonstration of manual integration using Isoprime software, but was at least temporatily thwarted by complaints about the version, being 167-4 instead of the 167-2 installed at LNDD. It remains to be seen if that is critical.

Absent live software, Davis drew some pictures showing the discretion available to operators, and described how each operation can affect the reported isotope values.

He noted the LNDD SOP says that manual adjustment may have significant affect on the reported CIR -- an important point in establishing a potential ISL violation.

Then he discussed what Mongongu and Frelat did while reprocessing the Stage 17 samples. He produced a table show operations they had done on each sample, being alterations from the "automatic" results. They were divided into categories, like "move peak ends", "add/delete background points", "reanalyse". In various cases the operators made nearly a dozen alterations to a single chromatogram -- all the time with the computed CIR value visible.

He was asked why it took so many steps, and he said it was so the operator would get the lines to fit as they liked -- which was described as "using my experience."

Davis made it clear he did not believe they were intentionally trying to achieve certain results. Rather, he felt they were honestly trying to fit the lines and peaks the best they could.

Did he reach a conclusion about their ability and competence to operate the instruments? Yes, he did. They clearly did not understand the instrument. He helped them load software onto the machine, and they didn't have a general idea how the software worked.



Davis was asked about Herr Doktor Professor Wolfram Meier-Augenstein's computed chart of retention time differences between the GC/MS and IRMS parts of the A and B sample analyses. He agrees it makes it difficult if not impossible to have identified the right peaks.

Suh runs Davis through a bunch of chromatograms, some bad, some very bad. He's asked about the isotropic composition of an area he says is called a "squashed hedgehog' in the background, and says it is unknowable.



At which point we adjourned, with Dunn looking grim, and Brenna not looking happy either.

We're still missing some pieces -- what is going on with the misplaced/substitued pages? Does a leak account for the sloping baselines?

We also have the results from the retesting to address. I can imagine Suh sarcastically asking, "Ignore the pressure, magnet, and peak identification problems. There's no manual processing there, so don't they show he's guilty?"

So we return tomorrow.

17 comments:

Bijoy said...
This comment has been removed by a blog administrator.
Anonymous said...

Wow, TBV, that's thorough. Don't you ever sleep? Thanks for your work. Just a little longer!

Anonymous said...

That's an awsome analysis. There should be a Pulitzer for this.

-Skip

Anonymous said...

You MUST be acknowledged for this fantastic work, under time and pressure. Are there Blog Oscars?? When do you start making the big money? I think Judge Hue has groupies (I'm one of them!) rather he 's ready or not! strbuk and Marc have done a great job of holding down the fort, and should not be left out of the praise!! AWESOME WORK!!

Anonymous said...

For his cross if Mr. Davis can take a cue from Dr. M-A, and learn to say "My Dear Friend" in just the right tone of voice, I think it is all over for the USADA.

-wds

Anonymous said...

Wonderful summary; it cleared up some things for me and provided clear corrections to a couple items I had gotten wrong on the fly during the day.

All of you have done work well above and beyond what could be expected of any (even obsessed) volunteer. I will be sincerely surprised if this is not noted in some history of the media impact of the internet. If there aren't any awards, then there should be.

I await tomorrow with a decent dose of tension - will Davis get to do his demonstration? Will Landis attempt to press a vote on the shift of the burden at the close of his case? Will an effort be made to clarify Bostre's role in deliberations? What games will be played with the clock and if the day runs out before closings, how long will we have to wait for them?

The circus train is rolling toward the station and there may be surprises in store. Not the least of which may be that Wed slides slowly to an end without any fireworks, monkeys, blind squirrels or admissions of personal tragedies. An aside for Bill: Although I grew up in upstate New York I picked up a variation on your phrase, probably in the military: even a blind pig finds the occasional acorn.

Thank you
Pcrosby

Anonymous said...

TBV:

If this is the kind of summary you provide on a voluntary basis, what kind of detail do you put into your work if you were paid?

Do you know how to manage? Someone may be looking for a new manager soon.

Here's an early acknowledgement to you and the others on the excellent work that is being put into this blog, informing, educating and entertaining those that are truly interested, open minded, and believe in innocence until proven otherwise.

Anonymous said...

you3 said...
TBV,

Your coverage (and Bill's) of the hearing has been better than any than any of us could have hoped for. You are responsible for making this a truely public hearing.

If Davis gets to demonstrate that by picking other reasonable baseline adjustment points and integratoin points for 5A-diol, the new dC13 measurement of 5A-diol is below the threshold of a positive finding, then the hearing pannel will find no comfort in the USADA arguments. Let's hope that such a demonstration is allowed today.

Anonymous said...

Trislax here:
First, great job TBV!
Next, a prediction that others have alluded to (or outright said) that I will repeat: The train station that we seem to be pulling in to is the "hang LNDD out to dry" station, where USADA concedes mistakes by the lab such that they cannot uphold sanctions, thereby keeping face while lambasting a poorly run WADA lab. Everyone wins. USADA can claim that their hands were tied by the shoddy work, WADA gets to clean house as LNDD, Landis gets off on a technicality, but gets off and takes a major hit in PR, which is why they crossed so personally yesterday (ruin his rep if you can't sanction). *sigh* Just my suspcions that a 2-1 for Landis, with appeal that will get upheald, but Landis loses in the long run through PR and suspicion, and WADA and USADA win by claiming poor labwork led to the technicality.

Anonymous said...

I, like many, came into all of this hoping that Landis is innocent, fearing that he might not be, believing that Landis was not getting "due process," however defined in this situation, and therefore simultaneously hoping that he would get off and fearing that doing so on a "technicality" would still leave open the possibility or likelihood of his guilt.

But yesterday was truly horrifying. The USADA's cross of Landis demonstrated to me a growing disinterest in focusing on issues germane to whether or not he doped, i.e. the science, and turning to whatever mud they could rake to bring him down. I'm disappointed in the whole LeMond thing (actually, I'm pretty disappointed in LeMond, too), but being an ass, let alone a friend or employer of an ass, doesn't make you a doper.

And then Davis' testimony -- of all the science we've heard, the fact that the lab techs simply didn't know how to operate the machinery they needed to use? That they were reconstructing baseline settings each time through, in an effort to get consistent results? In my line of work, that's called fitting the facts to meet your conclusions, whether intentionally or subconsciously (and frankly, we are all susceptible to the latter...). In my book, whether Landis actually doped or not, this is not a technicality, this is evidence that there simply is no evidence, because none was ever produced.

Anonymous said...

Key natural laws that come to mind in this case:

1. First draw the curves, then plot the data.
2. If it jams, force it. If it breaks, it needed replacing anyway.
3. The organization of any bureaucracy is very much like a sceptic tank. The really big chunks always rise to the top.

Anonymous said...

This is truly sad.

LNDD says, he's guilty because we say he is, off with his head!

USADA says, evidence, we don't need no stinking evidence.

The press says, he has to be guilty because everyone's doing it. Besides he's just a bad guy.

The general public, as always doesn't take the time to think, and jumps on board.

This is exactly what is wrong with our society today. They are the experts, of course he is guilty.
They seem to believe that there has never been a lab technician that has ever made a mistake or misused an instrument. It reminds me of the commercial where the patient mixes up the blood samples while in the room. Then they show the doctor telling an 80 year old man and women that they are both pregnant!

Anonymous said...

TBV,

Great job again.

I am continualy amazed by how bad the WADA Athelet distruction system has been set up. LNDD is a bad apple that must be thrown out, unfortunately it has already spoiled the whole crate. I think the French open taking their testing out of country is a key signal that athletic federations are steping back from LNDD, with good reason. I'm still concerned they will hang floyd by saying the evidience of incompetent work is irrelevent, we suspect Floyd a doper, we have results (forget its flawed, just look at the dancing monkey isn't he cute) that says he's a doper so he must be hung out to dry. regarding USADA and TT saying wait for the hearing before judging, well you had the smoke and mirrors TT. Not to mention you've been siping your own coolaid havent you? USADA has no desire to search for truth as evident by the case and the circus that they brought apon it. I feel bad for any athelete that has to go throug such a corupt system.

Atown, Tx.

Anonymous said...

TBV team:

Excellent summary! Excellent coverage. The Hearing has been a mulitstage race of its own magnitude dealing with science, leagal issues, government agencies and politics. Throw on the "entertainment" layer and there is something for everyone.

Thank you for sharing your insights.

Anonymous said...

What I find most troubling about the way this has played out is the complete and utter abandonment of the scientific portion of the case after the Lemond scandal. USADA has seemingly said, "Ok, forget about the lab, the errors, the untrained workers, the screwed up machines, the railroad job with the repeated l(')e(quippe)aks, the unbreakable WADA Omerta and the un-ending disregard for rules in purported search for the truth." What I hear them now saying is, "Mr. Landis, you wore black and your buddy drank-and-dialed Mr. Lemond, surely you are a doper!"

Once the Landis team rolled out Goldberger (with his precise lab philosophy and solicitation from UCLA Lab), Wolfram M-A (he of astute knowledge of Isotopic Applied Science), Dr. Amory (who serves on USADA Review boards and is a leader in the andrologic field) and then Simon Davis (who’s capacity to explain the IRMS method and machines borders scary), USADA has been left with one dog in this fight: attack the character of Mr. Landis.

If wearing all black or knowing your manager crank-called Lemond are ISL violations, then I’d like to see the part of the WADA Code where it is specified. Should it turn out that Landis is perceived as a “jerk” as a result of his public attempt to clear his name, I would bet dollars to doughnuts he’d rather be seen as an honest, un-doped jerk, than a nice-guy, huggable cheater. Perhaps that has been the problem in the past with the TH’s and the JA’s and the IB’s.

USADA has done itself a disservice by putting on the blinkers and shifting over into the slimy lane. Taking the high-road is a tried and true way to appear honorable. I think if they thought they still had a valid case, we’d see USADA in high-gear, pedal on the floor above the tabloid fodder.

Anonymous said...

I cannot find the words to say how completely I admire and appreciate your coverage of this case, as well as that of Judge Hue and Marc in Paris. I called to ask about submitting this coverage for the Pulitzer Prize, but they will only take submissions associated with a newspaper. The depth, quality, and timeliness of your work surpasses anything any newspaper would ever attempt. I, and I am sure others, will continue to attempt to find a suitable award for this monumental effort on yur part. Once again, thank you.

bamalaw

Anonymous said...

Goldberger is a Florida cocaine consultant for football. This is a steroid case.

Does Landis abuse cocaine and Pot Blege too.

Maybe that was why Landis wanted an inexperienced and paid witness to defend his doping crimes?