Document:Lauritsen Validation

From AIDS Wiki

NOTWITHSTANDING ANY OTHER NOTICE ON THIS PAGE, the material on this page is NOT available under the GNU Free Documentation License; in accordance with Title 17 U.S.C. section 107, it is posted in the manner of bulletin boards in schools and workplaces, to encourage public education and citizen awareness, without profit or payment, for persons and entities engaging in non-profit research and educational activities and purposes only.

Rethinking AIDS
May 1993
[revised 24 June 2006]

Within a month of each other, two reports appeared that claim to refute Peter Duesberg's Risk-AIDS hypothesis on the basis of "cohort" studies. One, by M.S. Ascher et al., rebukes Duesberg in the final paragraph: "The energies of Duesberg and his followers could better be applied to unravelling the enigmatic mechanism of the HIV pathogenesis of AIDS." (M.S. Ascher et al., "Does drug use cause AIDS?", Nature, 11 March 1993)

The other, by Martin T. Schechter et al., also rebukes Duesberg in the final paragraph: "It is a disservice to the many people infected with HIV-1 and a hindrance to public health initiatives for scientists to claim that HIV-1 is harmless and not aetiologically related to AIDS." (Martin T. Schechter et al., "HIV-1 and the aetiology of AIDS", Lancet, 13 March 1993)

Both studies are highly implausible – if for no other reason, because they show that drugs don't do anything. But drugs are not sugar pills, and taking them has physical consequences. Drugs, unlike HIV, are highly active from a biochemical standpoint.

Both studies are forms of survey research, my profession since 1966. In a letter to me (14 April 1993), Martin Schechter denied that his work represented survey research, but this merely indicates his ignorance of basic concepts. In survey research, data from a selected sample are projected to represent a greater universe or population. As explained in one of the classic texts of my field:

"Sampling, as probably everyone knows, arises from the impossibility or impracticability of studying an entire population. It is not very feasible, if at all possible, to study the entire population of the United States at a given time, nor is it necessary to test the entire contents of a well-sifted grain barrel to determine its quality content. Even where it is advisable to study an entire population, time and cost elements are usually prohibitive. Essentially, sampling is a problem in inference, the aim being to secure sufficient information from a representative segment of the population to enable one to infer the true state of affairs with respect to the characteristics under observation for the entire population within a certain range of error." (Robert Ferber, Statistical Techniques in Market Research, New York 1949)

After reading the two brief reports, I had grave doubts that the researchers understood how to design, conduct, or analyze survey research. Before doing an evaluation of the study myself, it would be necessary for me, as a professional analyst, to satisfy myself that the data were valid. Therefore, I wrote to both Ascher and Schechter, asking permission to look at raw data, questionnaires, and other study materials. Ascher did not reply. Schechter wrote:

Unfortunately, your request to inspect our raw data and other documents is problematic. As you are no doubt aware, there are tremendous concerns surrounding confidentiality in studies of this type. In our informed consent, we have specifically promised not only all the participants but their practitioners that the data we collect will not be seen by any individuals or agencies outside the investigators involved in the study. To allow anyone else to inspect the raw data would constitute a breach of this fundamental promise. (letter of 14 April 1993)

Schechter ignored the most specific request I made in my letter: a copy of the self-administered questionnaire he mentioned in his article. I fail to see how the release of a blank questionnaire – or of data, consisting of grouped numbers – could violate promises of "confidentiality". Apparently Schechter expects us to accept his research on faith – in effect, to buy a pig in a poke.

One of the cardinal principles of science is openness, which means sharing data and describing methodology in sufficient detail that a study could be replicated or in some other way verified. Although replication might not be possible or practical, there is another way the worth of the data could be evaluated: through validation (a term that has a special meaning in survey research). To validate a study means to take whatever steps are necessary to ensure that the data are good.

In professional survey research, it is the practice to validate all studies, using sophisticated techniques. Let's suppose that an in-person survey was conducted by local interviewers in a dozen cities around the country. After the questionnaires have come back from the field, interviewers from the home office, using WATS lines, validate a percentage of each interviewer's work. This means calling respondents and asking a few questions designed to ascertain that they really were interviewed, that they were eligible for the study, and that the rules of the study were followed. If even a single questionnaire fails the validation test, then validation is performed upon 100% of that interviewer's work. In the extremely rare event that cheating is discovered, the culprit is severely punished.

An unvalidated survey has little or no credibility. Since neither the Ascher nor the Schechter study is even open to validation, they deserve to be rejected on this basis alone. In addition, neither author describes the characteristics of the samples, so one has no idea how representative they might be of the populations from which they were allegedly drawn.

The editor of Nature, John Maddox, did not grant Duesberg the right to reply to the attack that was made on him. However, a letter from Duesberg, "HIV and the aetiology of AIDS", was published in Lancet. (Peter Duesberg, "HIV and the aetiology of AIDS", Lancet, 10 April 1993) In it, Duesberg thoughtfully dissects the Schechter study, demonstrating that it provides no proper controls, fails to quantify drug use, and ignores AZT treatment. Duesberg then criticizes the Ascher study for presenting information on drug use selectively, and for ignoring AZT treatment. He raises the serious charge that Ascher et al. had put together a phoney graph:

"Based on tables I and II the category 'seropositive/no drug use' is an empty set representing nobody. The curve, clearly labelled 'seropositive — no drug use', is therefore a fabrication." (Duesberg, letter cited.)

Duesberg turns Schechter et al.'s rebuke on its head: "I hope the drug hypothesis will become a hindrance to the physiologically (zidovudine) and psychologically toxic (positive AIDS test) public health initiatives of the unproven HIV-AIDS hypothesis." (Duesberg, letter cited)

Unlike Duesberg, I have refrained from analyzing either study, because I have no confidence in unvalidated data. In addition, the supposed findings of Ascher, Schechter et al. are unbelievable prima facie. It has been my experience as an analyst, without exception, that when data don't make sense, it is because there is something wrong with them. It doesn't make sense that a single, biochemically inactive virus could be the cause of the 29 (at last count), highly heterogeneous AIDS-indicator diseases. It doesn't make sense that drugs don't do anything.

[NB: See also Serge Lang's files on the Ascher paper.]

© 1993/2006 by John Lauritsen
Originally published in Rethinking AIDS, 1993