Hearing - Tue: Today's Davis, digested

Now I know how reporters feel towards the end of an Olympic games, contemplating writing their wrap-ups and retrospective pieces, while there are still events being played out.

Today we had a complete waste of time in USADA's cross-examination of Landis, which went essentially nowhere, in desultory fashion, in front of lots of cameras. Depending on whose account you read, Landis was either tense or surprisingly relaxed. It doesn't appear to us there's anything of particular use to USADA in it, though we'll see if there's mud thrown in closing arguments. Dance, monkey, dance.

---

We are in the middle of something far more germane to the case, being the direct examination of Simon Davis, Landis' expert on mass-spectrometry.

Davis is the kind of scary-good expert Landis had in Goldberger for the TE and Herr Doktor Professor Wolfram Meier-Augenstein for the IRMS. He's confident and able to explain what he's talking about clearly and able to answer questions without evasions, and he wants data, not "close enough"

He helped develop one version of the Isoprime used at LNDD; he's the founder of another Mass spec instrument company and helps design the instruments. Nuts and bolts reality. And he was present at LNDD for both the retesting of alternate B samples and reprocessing of data associated with the S17 tests at issue in the case.

He started out by explaining the theory of carbon-isotope-ratio analysis, and explained sample flow through an IRMS spectrometer -- the latter being something no one had done before. He was able to explain the retangular peaks in some spectrograms as being jets of reference gas -- something no one had done before. He explained the burn-off of volatile solvents into the "FID" as a function of a valve, which explains the flat "front-porch" of most IRMS spectrograms -- which no one had done before.

When Brunet asked about the difference between a pinial and a penning gauge reading, he was able and cheerfully willing to explain the multiple stage pumps and turbines and how one gauge read one stage, and the other the pressure at another location. When the USADA brief talked about an indicator light that showed "pressure ok", Davis showed a picture of it being a pilot light in inside of the machine, reflecting the first stage pump operation, and not the actual pressure at all.

But we're getting ahead of ourselves.

We've been waiting all hearing for his appearance with the kind of details we're getting. We know the reported results look kind of funny (see Amory), but we haven't known why. Davis is the guy who is telling us, and so far he's given parts of the story, but not quite yet a conclusion.

---

After various preparatories, he explained what goes on in analysis. Given the raw data, one identifies peaks, subtracts extraneous background, does integration, and computation to produce the isotope ratio. The raw data is the "zero" stuff that made no sense to Brenna during the reprocessing. (We expect this thread to come back later, but it isn't yet explained.) Peak identification and background subtraction can be done either automatically, or manually adjusted.

Davis has methodological problems with manually adjusted values, because it is "without process" to make it repeatable. There are as many different processes as there are analysts, and analyst moods. While a good analyst can sometimes improve quality, it's impossible to properly resolve inadequately separated materials -- hence Suh's repeated insistence on "good chromatography".

Then Davis noted that the three operators at the lab didn't seem to know their software very well. In moving files form one machine to the other, it became apparent they'd never done it before, as he had to show them how to do certain things. They expressed appreciation at being able to do it, because now they could use the Masslynx software on samples obtained on the earlier isoprime. The Masslynx is said to be easier in some ways to use.

While Mongongu and Frelat did analysis, he observed them doing many manual steps, while taking notes about what they were doing, and asking questions. This had been annoying to Mongongu, who complained about his asking why she was doing certain things. He observed them moving peak starts and stops, and adding and removing data points to the background trace. When asked what they were doing, they'd reply, "using my experience", which doesn't seem like a repeated methodology.

All the while doing these manual adjustments, the operators were seeing the resultant CIR values. It is obvious, but so far unstated, that in doing so it is trivial to adjust starts, stops and the background to obtain a value that "seems correct"

At this point, Davis illuminated a point that had escaped USADA and Dr. Brenna: the software had a "save method" function by which the starts/stops and background selections made manually can be saved. When loaded with the sample data, one can repeatedly analyse the samples and get identical results. This was news to the staff. Their manual reprocessing had started from scratch each time, not from decisions saved from the original analysis.

So, while USADA, LNDD, and Brenna seemed to be fat dumb and happy getting numbers "close" to those of the original analysis (except when they weren't), Davis showed they should have been able to get identical numbers with no effort -- if they knew the software and were interested in repeatable results.

---

Then Davis skipped to tales of his observations while present during the retesting of B samples from other stages. Asked point blank if he thought the results were reliable and accurate, he said they were totally unreliable, and started into reasons.

He began here with a discussion of linearity. Linearity is the ability of the instrument to make proper isotope measurements for large peaks as well as small ones. It is typically measured by injecting a reference gas with known composition into the machine at varying pressures, resulting in varying size peaks. They should analyze to the same carbon ratio no matter the pressure. If the machine is operating correctly, it should give good results over the machine's specified range of pressures -- not the expected range of the samples, but the machine range. If it's not linear in the design range, he says the machine is broken. He pulled up a copy of the Isoprime manual showing the procedure and design range. Asked if he was familiar with the manual, he said he'd written parts of it.

We learned that LNDD does not have a copy of the Isoprime manual. We've heard the machine was delivered with a manual for an earlier, similar machine, the Isochrome instead. But Davis wasn't allowed to see it when he was there.

[It's not clear to me if there are non-linearity corrections built into any of the software; I'd kind of expect that the results of a linearity run could be saved to do corrections, but I have no idea if is done.]

Davis noted the Isoprime was a somewhat notoriously non-linear machine, for which frequent linearity checks ought to be run. He was surprised to find LNDD reluctant to do it as often as he had requested. when he asked Dr. Aquilera if he was being reasonable, Dr. Aquilera said "no comment", whatever that meant.

In any event, the LNDD SOP requires linearity runs only once a month. But, Davis noted, they may not do it that often. Buisson, the supervisor, admitted to some linearity problems, resulting in their testing the instrument at restricted ranges, presumably because they couldn't get it to work right full range.

Then he pulled up the SOP, which documented "once a month" or more frequent linearity runs. Pulling out other documents, he showed that linearity appeared to have been done less frequently, sometimes seeming to skip a month.

This SOP mistake is an ISL violation that could flip the burden of proof; it may also affect analytical accuracy.

Davis is of the belief this Isoprime drifts in and out of linearity, and that the result is changes in reported isotope ratio that depend on peak height, not material composition. That is wrong.

---

Next, Davis had a look at the "isoprime2", the new machine used on the retesting. He produced a picture from his cell phone, showing "Micky Mouse Ears" on the machine. These are "lifting rings", on the critical high precision magnet that are only supposed to be there during installation and maintenance on the magnet. They are not supposed to be there during operation. When present, the effect is not unlike putting a speaker near a color-TV screen, where the picture gets distorted. In the machine, the magnetic field that is machined to 0.01mm tolerance may be deflected, leading to unpredictable results. (This might account for some of the machine instability).

---

Then, Suh had him address an assertion in the USADA brief about a pressure measurement. The brief made it appear like an "idiot light" on the dashboard of a car. Davis showed a picture of the light in question being inside the instrument, not on the operator panel, suggesting the person who wrote the response had "a complete lack of understanding of the instrument."

---

Davis was asked about the importance of having and using procedures documented in the manual. Davis said this was essential, because the usual operators are not expert on the machines, and must rely on the decades of experience distilled in the wisdom of the manuals. In particular, advice on operating pressures are very important.

We were shown pages from the reports of the B samples where the pressures were outside the required range of the machine. Davis said the value suggested undiagnosed leaks, leading to contamination and incorrect results.

Brunet asked questions about various guages, and Davis happily went into geek talk about the pumps and what the gauges were measuring when.

Tellingly, Davis said the screen shots of the various samples showed the pressure all over the place, which should not be the case; the value ought to be stable, he said.

There are many effects of wrong pressure, quite possibly bad linearity. Bad linearity and low pressure seem to go together at the LNDD.

---

Suh asked Davis about the importance of automatic sequencing. Davis started explaining it as a way of reducing operator error. A robot produces reproducible results. By way of example, they pulled up USADA 155, the sequence chart of one of the A sample tests. As well, they pulled out the result pages in the LDP from each stage of the sequence -- USADA 157, 163, 166, 169, 172, 177, 178, 179, 180, 181 and 183.

The way it's supposed to work is that you put all the vials in the autosampler, and press start; you come back and get your summary results and the individual pages. If the run doesn't complete, you don't get a summary. It if stops, its because the operator stopped it for some reason, and restarted.

This particular series of pages, presented in the LDP as the processing of a Landis sample, was determined by Davis to have pages from other runs. The summary sheet has a "batch name", here "23016"; some of the sample sheets match; others do not, and have blanks. Further, the reported analytic numbers on the sample sheets do not match those on the summary sheets.

The same is true of results on the B sample - sheets that do not appear to match the batch, and have incorrect reported analytic values.

This was left somewhat hanging -- we don't know what it means, but there is speculation it relates to undocumented "re-injections". There is nothing wrong with re-injections, Davis says, if they are documented. But none of these seem to be documented, so we don't know what is going on.

---

Davis next attempted to do a live demonstration of manual integration using Isoprime software, but was at least temporatily thwarted by complaints about the version, being 167-4 instead of the 167-2 installed at LNDD. It remains to be seen if that is critical.

Absent live software, Davis drew some pictures showing the discretion available to operators, and described how each operation can affect the reported isotope values.

He noted the LNDD SOP says that manual adjustment may have significant affect on the reported CIR -- an important point in establishing a potential ISL violation.

Then he discussed what Mongongu and Frelat did while reprocessing the Stage 17 samples. He produced a table show operations they had done on each sample, being alterations from the "automatic" results. They were divided into categories, like "move peak ends", "add/delete background points", "reanalyse". In various cases the operators made nearly a dozen alterations to a single chromatogram -- all the time with the computed CIR value visible.

He was asked why it took so many steps, and he said it was so the operator would get the lines to fit as they liked -- which was described as "using my experience."

Davis made it clear he did not believe they were intentionally trying to achieve certain results. Rather, he felt they were honestly trying to fit the lines and peaks the best they could.

Did he reach a conclusion about their ability and competence to operate the instruments? Yes, he did. They clearly did not understand the instrument. He helped them load software onto the machine, and they didn't have a general idea how the software worked.

---

Davis was asked about Herr Doktor Professor Wolfram Meier-Augenstein's computed chart of retention time differences between the GC/MS and IRMS parts of the A and B sample analyses. He agrees it makes it difficult if not impossible to have identified the right peaks.

Suh runs Davis through a bunch of chromatograms, some bad, some very bad. He's asked about the isotropic composition of an area he says is called a "squashed hedgehog' in the background, and says it is unknowable.

---

At which point we adjourned, with Dunn looking grim, and Brenna not looking happy either.

We're still missing some pieces -- what is going on with the misplaced/substitued pages? Does a leak account for the sloping baselines?

We also have the results from the retesting to address. I can imagine Suh sarcastically asking, "Ignore the pressure, magnet, and peak identification problems. There's no manual processing there, so don't they show he's guilty?"

So we return tomorrow.