Psyche, Science, and Society

Lancet editor on BBC report that Lancet mortality study “robust”

In response to yesterday’s report that the British government’s own advisers had told the Foreign Office that the 2006 Lancet mortality study was “robust”, Lancet editor Richard Horton wrote, in the Guardian’s Comment Is Free section:

Our collective failure has been to take our political leaders at their word. This week, the BBC reported that the government’s own scientists advised ministers that the Johns Hopkins study on Iraq civilian mortality was accurate and reliable. This paper was published in the Lancet last October. It estimated that 650,000 Iraqi civilians had died since the American- and British-led invasion in March 2003.

Immediately after publication, the prime minister’s official spokesman said that The Lancet’s study “was not one we believe to be anywhere near accurate”. The foreign secretary, Margaret Beckett, said that the Lancet figures were “extrapolated” and a “leap”. President Bush said: “I don’t consider it a credible report”.

Scientists at the UK’s Department for International Development thought differently. They concluded that the study’s methods were “tried and tested”. Indeed, the Hopkins approach would likely lead to an “underestimation of mortality”.

The Ministry of Defence’s chief scientific advisor said the research was “robust”, close to “best practice”, and “balanced”. He recommended “caution in publicly criticising the study”.

When these recommendations went to the prime minister’s advisers, they were horrified. One person briefing Tony Blair wrote: “are we really sure that the report is likely to be right? That is certainly what the brief implies?” A Foreign Office official was forced to conclude that the government “should not be rubbishing The Lancet”.

The prime minister’s adviser finally gave in. He wrote: “the survey methodology used here cannot be rubbished, it is a tried and tested way of measuring mortality in conflict zones”.

How would the government respond?

Would it welcome the Hopkins study as an important contribution to understanding the military threat to Iraqi civilians? Would it ask for urgent independent verification? Would it invite the Iraqi government to upgrade civilian security?

Of course, our government did none of these things. Tony Blair was advised to say: “the overriding message is that there are no accurate or reliable figures of deaths in Iraq”.

His official spokesman went further and rejected the Hopkins report entirely. It was a shameful and cowardly dissembling by a Labour - yes, by a Labour - prime minister.

Indeed, it was even contrary to the Americans’ own Iraq Study Group report, which concluded last year that “there is significant underreporting of the violence in Iraq”.

This Labour government, which includes Gordon Brown as much as it does Tony Blair, is party to a war crime of monstrous proportions. Yet our political consensus prevents any judicial or civil society response. Britain is paralysed by its own indifference.

At a time when we are celebrating our enlightened abolition of slavery 200 years ago, we are continuing to commit one of the worst international abuses of human rights of the past half-century. It is inexplicable how we allowed this to happen. It is inexplicable why we are not demanding this government’s mass resignation.

Two hundred years from now, the Iraq war will be mourned as the moment when Britain violated its delicate democratic constitution and joined the ranks of nations that use extreme pre-emptive killing as a tactic of foreign policy. Some anniversary that will be.

Add comment March 27th, 2007

British government expert said Lancet mortality study was “robust”

The BBC reports on emails they obtained through a Freedom of Information request regarding the British government’s response to the 2006 Lancet mortality study:

Iraqi deaths survey ‘was robust’

By Owen Bennett-Jones

BBC World Service

The British government was advised against publicly criticising a report estimating that 655,000 Iraqis had died due to the war, the BBC has learnt.

Iraqi Health Ministry figures put the toll at less than 10% of the total in the survey, published in the Lancet.

But the Ministry of Defence’s chief scientific adviser said the survey’s methods were “close to best practice” and the study design was “robust”.

Another expert agreed the method was “tried and tested”.

Mortality rates

The Iraq government asks the country’s hospitals to report the number of victims of terrorism or military action.

Critics say the system was not started until well after the invasion and requires over-pressed hospital staff not only to report daily, but also to distinguish between victims of terrorism and of crime.

The Lancet medical journal published its peer-reviewed survey last October.

It was conducted by the John Hopkins School of Public Health and compared mortality rates before and after the invasion by surveying 47 randomly chosen areas across 16 provinces in Iraq.

The researchers spoke to nearly 1,850 families, comprising more than 12,800 people.

In nearly 92% of cases family members produced death certificates to support their answers. The survey estimated that 601,000 deaths were the result of violence, mostly gunfire.

Shortly after the publication of the survey in October last year Tony Blair’s official spokesperson said the Lancet’s figure was not anywhere near accurate.

He said the survey had used an extrapolation technique, from a relatively small sample from an area of Iraq that was not representative of the country as a whole.

President Bush said: “I don’t consider it a credible report.”

But a memo by the MoD’s Chief Scientific Adviser, Sir Roy Anderson, on 13 October, states: “The study design is robust and employs methods that are regarded as close to “best practice” in this area, given the difficulties of data collection and verification in the present circumstances in Iraq.”

‘Cannot be rubbished’

One of the documents just released by the Foreign Office is an e-mail in which an official asks about the Lancet report: “Are we really sure the report is likely to be right? That is certainly what the brief implies.”

The reply from another official is: “We do not accept the figures quoted in the Lancet survey as accurate. ”

In the same e-mail the official later writes: “However, the survey methodology used here cannot be rubbished, it is a tried and tested way of measuring mortality in conflict zones.”

Asked how the government can accept the Lancet’s methodology but reject its findings, the government has issued a written statement in which it said: “The methodology has been used in other conflict situations, notably the Democratic republic of Congo.

“However, the Lancet figures are much higher than statistics from other sources, which only goes to show how estimates can vary enormously according to the method of collection.

“There is considerable debate amongst the scientific community over the accuracy of the figures.”

‘Mainstreet bias’

In fact some of the British government criticism of the Lancet report post-dated Sir Roy’s comments.

Speaking six days after Sir Roy praised the study’s methods, British foreign office minister Lord Triesman said: “The way in which data are extrapolated from samples to a general outcome is a matter of deep concern….”

Some scientists have subsequently challenged the validity of the Lancet study. Questions have been asked about the survey techniques and the possibility of “mainstreet bias”.

Dr Michael Spagat of Royal Holloway London University says that most of those questioned lived on main streets which are more likely to suffer from car bombs: “It would appear they were only able to sample a small sliver of the country,” he said.

Dr Spagat has previously conducted research with Iraq Body Count, an NGO that counts deaths on the basis of media reports and which has produced estimates far lower than those published in the Lancet.

If the Lancet survey is right, then 2.5% of the Iraqi population - an average of more than 500 people a day - have been killed since the start of the war.

The BBC World Service made a Freedom of Information Request on 28 November 2006. The information was released on 14 March 2007.

Now we have hard evidence, if any was needed, that the “Coalition” governments’ dismissal of the study was not based on any scientific qualms. When he spoke in Boston, Gilbert Burnham similarly told of a meeting at US AID that was intended to rubish the study, but did not find major faults.

[Thanks to Tim Holmes at Media Lens Massage Board.]

UPDATE:

Former British diplomat Craig Murray on this story:

Lying About The Dead

An extraordinary story appeared once this morning on BBC News 24, and then was buried.

The BBC World Service has obtained a leaked document. It is an official appraisal by British government scientists across government departments, commissioned by 10 Downing Street, of the study published by the Lancet that estimated 655,000 dead in Iraq. The appraisal says that the methodology is correct and that the study “follows best practice”.

Astonishingly, the official DFID verdict was that 655,000 dead is “If anything, an underestimate”.

Yet the Government poured scorn on the Lancet study, despite having commissioned a report from their own scientists that said it was good. Who can doubt that if the government scientists had rubbished the study, the number ten spin machine would have publicised that like crazy?

Doubtless the Official Secrets Act will be wheeled out to try and sit on the government scientists’ report, which the BBC already seems to have reburied, showing its typical craven attitude towards the Blair government.

Personally, I did not know how much credence to give the study published in the Lancet, not being technically equipped to evaluate it. We can now be confident that the death toll in Iraq was over 600,000 a year ago, and probably over 700,000 now.

There is much talk of Blair’s legacy. In fact he has two major legacies. 700,000 rotting corpses, and the culture of lies that sought to suppress the truth about it.

1 comment March 26th, 2007

Understanding research on cancer screening: Getting the numerator and denominator correct

Tuesday’s New York Times had a very nice story on lung cancer screening and how to reconcile two contrasting studies. One study showed that screening increased survival rates, while the other showed that screening did not reduce mortality (death rates). The paper is a nice example of the complexities involved incorrectly interpreting research.

How Two Studies on Cancer Screening Led to Two Results
By H. GILBERT WELCH, STEVEN WOLOSHIN and LISA M. SCHWARTZ

Of all the forms of cancer, lung cancer is by far the deadliest. So doctors have long hoped to come up with a screening test that would find it early, before it can grow and become untreatable.

Last fall, The New England Journal of Medicine published a study concluding that spiral CT screening (a kind of three-dimensional chest X-ray) would make most lung cancers curable. It sounded like wonderful news. For proponents of screening, it was a call to action: the Lung Cancer Alliance is starting an advertising campaign featuring sports celebrities trying to persuade you to make the “right call” and get screened.

But just last week The Journal of the American Medical Association published a study concluding that spiral CT screening is not only ineffective, but may actually be harmful, prompting unnecessary surgery that carries risks of its own.

How could these two studies — in the country’s two most prestigious medical journals — arrive at diametrically opposite conclusions? An answer requires a clear understanding of the goal of cancer screening.

That goal is to save lives — or, in scientific terms, reduce mortality. Simply finding cancer early is not enough.

Finding cancer early saves lives only if two conditions are met: the cancers detected are the ones that kill people; and early treatment prevents these deaths.

It is not enough to increase survival. While that may seem to be the mirror image of mortality, it can be a terribly misleading measure of the value of screening.

In the 1970s and ’80s, there was great interest in screening smokers for lung cancer using conventional chest X-rays. The question was seen as so important that it was examined using the gold standard of medical studies, a randomized trial. Half the participants were randomly selected to receive regular chest X-rays; half did not and served as the control group.

Three such randomized trials were conducted, and all three showed that screening did not reduce mortality. In fact, two reported slightly higher death rates in the group receiving chest X-rays.

The most famous of these trials, at the Mayo Clinic, showed how misleading survival can be. Although the 10-year survival rate doubled with screening, mortality was not reduced; indeed, screening may have increased it. The Mayo trial also showed that more than a decade after screening was stopped, there were still more cancers in the screened group. This shouldn’t happen: in two large randomly selected groups, there should be the same number of cancers in both. The chest X-rays must have detected some lung cancers that were never destined to cause symptoms or death — a phenomenon known as overdiagnosis.

This phenomenon challenges our conventional view of cancer as an inexorably progressive disease. Research in screening has demonstrated that what pathologists call cancer encompasses a broad spectrum of disorders: some cancers rapidly progress to death, some do so more slowly, and some don’t progress at all (or may even regress).

Overdiagnosis is even more of a concern for spiral CT, because it can detect far more abnormalities than chest X-rays. In fact, a screening program in Japan found about 10 times as many lung cancers with spiral CT as had been found in the same population using chest X-rays. More remarkably, the chance of having lung cancer detected by spiral CT was almost the same in nonsmokers and smokers.

This flies in the face of everything we know about lung cancer and smoking — 50 years of research showing that smokers are 10 to 20 times as likely as nonsmokers to die from lung cancer. This is powerful evidence that spiral CT detects some lung cancers that will never affect patients.

Because all lung cancer patients get treated, overdiagnosis means some people receive treatment that can’t help them (because they do not need it) and can only cause harm. Most patients given diagnoses of early lung cancer undergo surgery to remove part of a lung, a major operation from which about 5 percent die within a month.

With this background, let’s look at the two recent studies on screening.

The New England Journal study reported screening about 31,000 people with spiral CTs and finding 484 with lung cancer. These patients had a 10-year survival of 80 percent — compared with 10 percent for current lung cancer patients in the United States. The JAMA study reported screening about 3,200 people and finding 144 with lung cancer. (The detection rate was higher because this study had older patients and longer follow-up.) Of 3,200 people, 38 died from lung cancer — the same mortality rate expected for people of similar age and smoking history in the absence of screening.

In short, The New England Journal reported increased survival; JAMA reported no difference in mortality.

Most of us interpret “increased survival” to mean fewer deaths. But it does not, because survival is subject to two powerful distortions.

The first is called lead-time bias. Simply advancing the time of diagnosis (as with CT screening) will always increase survival.

Imagine two patients with lung cancer. Even if both die at age 70, a patient with cancer diagnosed by spiral CT screening at age 59 has a longer survival than one with cancer diagnosed because of symptoms (cough, weight loss and so on) at age 67. The first patient survives 11 years; the second 3 years. But both died at the same age. Survival is increased, but mortality is the same.

A second source of distortion results from overdiagnosis, when screening finds cancers that were never destined to progress and cause death. Overdiagnosis bias can also drastically inflate survival statistics, even if mortality is unchanged.

To understand why, you need to understand the definition of the two statistics. Both are fractions. Survival is calculated over a fixed period, for example 5 or 10 years.

Overdiagnosis inflates both the numerator of the survival statistic (number alive at a specified time) and the denominator (number of diagnoses). For the mortality statistic, overdiagnosis has no effect on the numerator (number of deaths) or the denominator (number studied). Perhaps the easiest way to understand this is to imagine if we told all the people in the country that they had lung cancer today: lung cancer mortality would be unchanged, but lung cancer survival would skyrocket.

The goal of lung cancer screening is to reduce mortality — to save lives. Because the New England Journal study examines only survival, it cannot tell us whether any lives are saved. Because the JAMA study examines mortality, it is the more valid study. It also corroborates the Mayo trial finding that a significantly increased survival rate can coexist with no difference in mortality.

The JAMA study also highlights the tradeoffs involved in lung cancer screening. The findings show that compared with no screening, if 1,000 people are screened over five years there would be 48 more lung cancer diagnoses, 46 more lung cancer operations (which would be expected to cause 2 deaths) and no lung cancer deaths prevented. The study data are consistent with as many as eight deaths prevented by screening, or eight extra deaths caused by it.

But neither study is definitive, because neither was a randomized trial. And both required assumptions. Given the potential benefit (so many people die from lung cancer) and the potential harms (some die from treatments), no one should have to assume anything.

Luckily, two randomized trials are under way — one a Dutch-Belgian collaboration, the other sponsored by the National Cancer Institute. Recent experience, notably with hormone replacement in postmenopausal women, has demonstrated how presuming benefits in the absence of randomized trials can cause real harm. To avoid repeating these mistakes, we should not screen for lung cancer unless the trials demonstrate a reduction in mortality.

This essay is by H. Gilbert Welch, Steven Woloshin and Lisa M. Schwartz. Dr. Welch is the author of “Should I Be Tested for Cancer? Maybe Not and Here’s Why” (University of California Press). He, Dr. Schwartz and Dr. Woloshin are senior research associates at the VA Outcomes Group in White River Junction, Vt.

1 comment March 15th, 2007

Psychological torture is torture, new study finds

A new study in the March Archives of General Psychiatry examines the mental health consequences of the varied psychological manipulations which are commonly referred to as psychological torture., as compared to physical torture The equivalent effects of these two forms of abuse undercuts the Bush administration claim that these psychological manipulations are not torture and are not banned.

Here is an AFP article on the study:

US has no case for redefining torture: study

Psychological torture, including some of the techniques reportedly used on Guantanamo Bay detainees, appears to inflict the same kind of long-term mental damage as physical abuse, a study released Monday said.

Researchers who evaluated the mental health of soldiers and civilians tortured during the 1990s Balkan wars found that victims of psychological abuse were just as likely to suffer from post-traumatic stress disorder (PTSD) and depression as victims of classic physical torture methods.

The researchers also reported that the torture victims rated some techniques such as stress positions, isolation, sleep deprivation and blindfolding as distressing as most physical torture methods.

“Ill treatment during captivity, such as psychological manipulations, humiliating treatment, and forced stress positions, does not seem to be substantially different from physical torture in terms of the severity of mental suffering they cause,” the study’s authors wrote.

“Thus, these procedures do amount to torture, thereby lending support to their prohibition by international law,” they wrote in the journal of the Archives of General Psychiatry.

The investigators said their findings undermine moves by the US government to narrow its definition of torture in order to free interrogators to use certain psychological methods aimed at breaking a prisoner’s resistance.

In 2003, lawyers for the US Justice Department and a Pentagon working group report on detainee interrogations made the case for a narrow definition of torture that excludes procedures such as blindfolding and hooding, forced nudity, isolation and other psychological manipulations.

The Justice Department memorandum argued that the scope of the term torture should be limited to those acts which could be shown to result in “prolonged mental harm,” according to the study.

The development followed allegations of human rights abuses at US detention facilities in Iraq, Afghanistan and Guantanamo Bay, Cuba.

However, the authors of this paper said that based on their analysis of the experiences of torture victims from the modern Balkans conflict, the US appears to be drawing a distinction without a difference.

They said their analysis of 279 Bosnian, Croatian and Serb torture survivors showed that the individuals who suffered psychological abuse had the same rates of depression, PTSD, and social and work-related problems as others who had endured beatings, burnings, sexual abuse and other forms of physical punishment at the hands of their captors.

They suggested that the trauma is the same, because regardless of the form of aggression, the effect is to create fear or anxiety in the detainee while at the same time removing any form of control from the person in order to create a state of total helplessness.

“The distinction between torture and degrading treatment is not only useless, but also dangerous,” said Steven Miles, professor of bioethics at the University of Minnesota, Minneapolis, in an accompanying editorial in the journal of the Archives of General Psychiatry.

The study was written by Metin Basoglu, head of trauma studies at the Institute of Psychiatry, King’s College, London, with help from colleagues at the department of psychiatry at the Clinical Hospital Zvezdara in Belgrade.

Here is the Abstract of the study:

ABSTRACT

Context After the reports of human rights abuses by the US military in Guantanamo Bay, Iraq, and Afghanistan, questions have been raised as to whether certain detention and interrogation procedures amount to torture.

Objective To examine the distinction between various forms of ill treatment and torture during captivity in terms of their relative psychological impact.

Design and Setting A cross-sectional survey was conducted with a population-based sample of survivors of torture from Sarajevo in Bosnia and Herzegovina, Banja Luka in Republica Srpska, Rijeka in Croatia, and Belgrade in Serbia.

Participants A total of 279 survivors of torture accessed through linkage sampling in the community (Banja Luka, Sarajevo, and Rijeka) and among the members of 2 associations for war veterans and prisoners of war (Belgrade).

Main Outcome Measures Scores on the Semi-structured Interview for Survivors of War, Exposure to Torture Scale, Structured Clinical Interview for DSM-IV, and Clinician-Administered PTSD (posttraumatic stress disorder) Scale for DSM-IV.

Results Psychological manipulations, humiliating treatment, exposure to aversive environmental conditions, and forced stress positions showed considerable overlap with physical torture stressors in terms of associated distress and uncontrollability. In regression analyses, physical torture did not significantly relate to posttraumatic stress disorder (odds ratio, 1.41, 95% confidence interval, 0.89-2.25) or depression (odds ratio, 1.41, 95% confidence interval, 0.71-2.78). The traumatic stress impact of torture (physical or nonphysical torture and ill treatment) seemed to be determined by perceived uncontrollability and distress associated with the stressors.

Conclusions Ill treatment during captivity, such as psychological manipulations, humiliating treatment, and forced stress positions, does not seem to be substantially different from physical torture in terms of the severity of mental suffering they cause, the underlying mechanism of traumatic stress, and their long-term psychological outcome. Thus, these procedures do amount to torture, thereby lending support to their prohibition by international law.

Perhaps I’ll write more on this study later, when I’ve better digested it.

UPDATE: This article is available online here. Unfortunately, an Editorial Commentary by Steven Miles — Science and Torture — is apparently not available to non-subscribers. I will see if I can get a copy and make it available.

7 comments March 5th, 2007

Nature on Iraq mortality study

The British journal Nature has a story on the controversy around the Burnham et al. 2006 Lancet Iraq mortality study:

Death toll in Iraq: survey team takes on its critics

Jim Giles

Raw data should settle arguments over study methods.

It’s not often that George W. Bush takes time out to attack a scientific paper on the day that it’s released. But then few papers attract as much attention as the one that claimed that more than half a million people, or 2.5% of the population, had died in Iraq as a result of the 2003 invasion. Published last October in the run-up to the US mid-term elections, the interview-based survey attracted huge press interest and controversy.

The media spotlight has moved on, but interest within the scientific community has not. The paper has been dissected online, graduate classes have been devoted to it and critiques have appeared in the literature with more in press. So far, the discussion has created more heat than light. Many of the criticisms that dogged the study are unresolved. For example, Nature has discovered that different authors give conflicting accounts of exactly how the survey was carried out. And although many researchers say the questions hanging over the study are not substantial enough for it to be dismissed, a vocal minority disagrees.

The controversy creates extra interest in the authors’ decision, made last week, to release the raw data behind the study. Critics and supporters will finally have access to information that may settle disputes.

On paper, the study seems simple enough. Eight interviewers questioned more than 1,800 households throughout Iraq. After comparing the mortality rate before and after the invasion, and extrapolating to the total population, they concluded that the conflict had caused 390,000–940,000 excess deaths (G. Burnham, R. Lafta, S. Doocy and L. Roberts Lancet 368, 1421–1428; 2006). This estimate was much higher than those based on media reports or Iraqi government data, which put the death toll at tens of thousands, and the authors, based at Johns Hopkins University in Baltimore, Maryland, and Al Mustansiriya University in Baghdad, have found their methods under intense scrutiny.

Much of the debate has centred on exactly how the survey was run, and finding out exactly what happened in Iraq has not been straightforward. The Johns Hopkins team, which dealt with enquiries from other scientists and the media, was not able to go to the country to supervise the interviews. And accounts of the method given by the US researchers and the Iraqi team do not always match up.

Several researchers, including Madelyn Hicks, a psychiatrist at King’s College London, recently published criticisms of the study’s methodology in The Lancet (369, 101–105; 2007). One key question is whether the interviews could have been done in the time stated. The October paper implied that the interviewers worked as two teams of four, each conducting 40 interviews a day — a very high number given the need to obtain consent and the sensitive nature of the questions.

The US authors subsequently said that each team split into two pairs, a workload that is “doable”, says Paul Spiegel, an epidemiologist at the United Nations High Commission for Refugees in Geneva, who carried out similar surveys in Kosovo and Ethiopia. After being asked by Nature whether even this system allowed enough time, author Les Roberts of Johns Hopkins said that the four individuals in a team often worked independently. But an Iraqi researcher involved in the data collection, who asked not to be named because he fears that press attention could make him the target of attacks, told Nature this never happened. Roberts later said that he had been referring to the procedure used in a 2004 mortality survey carried out in Iraq with the same team (L. Roberts et al. Lancet 364, 1857–1864; 2004).

Other arguments focus on the potential for ‘main-street bias’, first proposed by Michael Spagat, an expert in conflict studies at Royal Holloway, University of London. In each survey area, the interviewers selected a starting point by randomly choosing a residential street that crossed the main business street. Spagat says this method would have left out residential streets that didn’t cross the main road and, as attacks such as car bombs usually take place in busy areas, introduced a bias towards areas likely to have suffered high casualties.

The Iraqi interviewer told Nature that in bigger towns or neighbourhoods, rather than taking the main street, the team picked a business street at random and chose a residential street leading off that, so that peripheral parts of the area would be included. But again, details are unclear. Roberts and Gilbert Burnham, also at Johns Hopkins, say local people were asked to identify pockets of homes away from the centre; the Iraqi interviewer says the team never worked with locals on this issue.

Many epidemiologists say such discrepancies are understandable given that Roberts and Burnham could not directly oversee the survey, and do not justify accusations that the process was flawed. For those who disagree, access to the raw data is essential. Although previously reluctant to release them, Roberts and Burnham now say they are removing information that could be used to identify interviewers or respondents and will release the data within the next month to people with appropriate “technical competence”.

One researcher keen to see the numbers is Spagat. The 2004 survey used GPS coordinates instead of the main-street system to identify streets to sample, and when Spagat used the limited data available so far to compare the two studies for the period immediately following the invasion, he found that the 2006 study turned up twice as many violent deaths, suggesting that main-street bias may be present.

Roberts and others question Spagat’s methods. But the issue could be checked using the raw data. If main-street bias exists, says Spagat, then death rates will fall as the interviews move away from the main street.

The raw data may also help address a fear that some researchers are expressing off the record: that the Iraqi interviewers might have inflated their results for political reasons. That could show up in unusual patterns within the data.

Roberts and Burnham say they have complete confidence in the Iraqi interviewers, after working with them directly for the 2004 study. And supporters say that criticisms should not detract from the fact that the Iraqi team managed to produce a survey under extremely difficult circumstances. Security threats forced the team to change travel plans and at one point to consider cancelling the survey altogether. Since its completion, one interviewer has been killed and another has left Baghdad, although it is not known whether either case is linked to their involvement in the survey. Either way, the continuing violence in the country is enough for the remaining interviewers to say that they are not willing to repeat the exercise.

Nature 446, 6-7 (1 March 2007) | doi:10.1038/446006a; Published online 28 February 2007

All in all, surveys are only as good as the field work. I do find the reports here that different authors have different accounts of how that field work was conducted to be disturbing, though not necessarily surprising. Having been in charge of teams conducting surveys, I’m aware that I often did not know all the details regarding field work. That was often entrusted to other colleagues. But I do think that these issues should be clarified, to the degree possible. Perhaps the US and Iraqi researchers could write a report describing the methods in greater detail.

I have to say that I’m rather dubious that the raw data will settle the questions. While secondary analyses can probably detect deliberate cheating, which I find rather unlikely in any case, I doubt it can settle issues regarding “main street” or other bias. I would doubt that the data, when identifiers are removed, are going to contain information on location. But the main street bias, if it exists, seems likely to be a somewhat minor issue. The only way it becomes major is through making a number of assumptions that seemed created for the purpose of justifying the importance of the bias.

I doubt that any such report will silence critics. Nor should it. Science advances by criticism. What is disturbing in this case is not the criticism, but the extent to which the study was criticized in public venues for issues that are typical of research studies of the type. The criticism, especially once they hit the press, have often been presented in a “gotcha” fashion, rather than as an attempt to understand the study and the phenomenon under examination, namely postwar Iraqi mortality.

Ultimately, as Burnhmam said when he spoke at MIT Tuesday, the answer is unknowable. Science advances by replication. In this case, I’m not sure that replication is possible. Personally, I would like to see a collaboration between the Lancet study team and Jon Pedersen, the director of the Iraq Living Conditions Survey, the only other epidemiologic survey to attempt to assess Iraqi postwar mortality. Having met and corresponded with all three of Jon Pedersen, Les Roberts and Gilbert Burnham, I have great respect for all three. Science in difficult conditions has sometimes advanced through collaboration of those with differing perspectives, or even “biases.” That way, they can control for any potential unconscious “bias.” [I rule out deliberate bias in this case.] Jon’s extensive experience conducting surveys in the Middle East could compliment Les and Gilbert’s experience with assessing mortality in conflict situations. Personally, I would love to be in on their discussions designing such a study! This subject would be a natural, should the conditions in Iraq ever be safe enough to allow further mortality studies.

Les Roberts, in particular, has said innumerable times that one way to check their findings would be for a reporter to go to a sample of graveyards and find out if the majority of contemporary deaths are from violence, as the Lancet results indicate, or from nonviolent causes as much lower estimates such as that from Iraq Body Count would suggest. I think this would be an excellent idea. It would give a sense of what ballpark we are in. I wish some of the Lancet critics would put a fraction of the energy they spend attempting to discredit the study into inducing a reporter to conduct this work. In a matter of days we could have much better sense regarding the broad range into which the Iraq violent mortality falls. Doesn’t any reader have contacts with one of the reporters over there?

In the meantime, short of being discredited, the Lancet study suggests that postwar mortality has been high. It is almost certainly over 100,000 and there is a reasonable chance that it is far higher. It is a catastrophe and a humanitarian disaster. Bush and Blair, and so many other, both Anglo-American and Iraqi, bear responsibility for unleashing hell upon the Iraqi people.

Thanks to Media Lens for this.

4 comments March 1st, 2007

Gilbert Burnham discusses counting Iraq dead at MIT

Gilbert Burnham, lead author of the 2006 Lancet Iraq casualty study. You can watch the seminar here.

I was there and felt he did a credible job presenting and defending the study. While there were quite a number of questions, there was no one among this highly qualified audience who thought the study was not highly credible.

A few of the details he presented reduced some of my qualms about the study, as did my brief discussions with him afterwards. He seemed to be a careful researcher and did not dismiss reasonable critiques.

One thing that Burnham revealed is that they hope soon to release the identifier-removed data to a select group of qualified researchers. It will help to have independent analyses, though I doubt that it will do much to calm the criticism.

In the end, as Burnham and Les Roberts have said from the beginning, what is needed is replication, should researchers decide that the dangers are worth taking in a climate that is even more violent than that when the study was conducted last summer.

11 comments February 27th, 2007

New article on Lancet study

The Johns Hopkins Magazine has an interesting article on the Lancet Iraq mortality study: The Number:

Burnham, who is professor of epidemiology and co-director of Johns Hopkins’ Center for Refugee and Disaster Response (CRDR), tried to keep attention focused on what he thought the public needed to understand. “I have one central message,” he says. “That central message is that local populations, people caught up in conflict, do badly. This is not a study that says, Ain’t it awful. This is a study that says, We need to do something about this.”

A message lost in a number.

Thanks to Tim Lambert for this.

Add comment February 12th, 2007

Psychodynamic therapy works for panic disorder

The New York Times today reported on a new study demonstrating that psychoanalytically-informed psychotherapy is effective for panic disorder. In fact, it had a very impressive response rate:

In Rigorous Test, Talk Therapy Works for Panic Disorder
By Benedict Carey

The field of psychoanalysis has struggled with a disabling internal conflict in recent years: whether to subject the therapy to rigorous testing, like the process through which new drugs are approved, or to insist that the insights it provides are self-evident and cannot be put under a microscope.

This internal debate has raged even as analysis, Freud’s open-ended talking cure, has become increasingly marginalized as a practice. But the ground rules may soon change.

Last week, a team of New York analysts published the first scientifically rigorous study of a short-term variation of the therapy for panic disorder, a very common form of anxiety. The study was small, but the therapy proved to be surprisingly effective in a group of severely disabled people.

The paper, which appeared in psychiatry’s flagship journal, The American Journal of Psychiatry, is one of the most significant steps in a small but growing effort to study how this so-called psychodynamic therapy works, and for whom.

The brand of therapy tested relies on core tenets of analysis, like the search for the underlying psychological meaning of symptoms. But unlike traditional psychoanalysis, it focused on relieving symptoms quickly, and was time-limited. Previous studies of similar approaches have shown some promise for other disorders, like depression.

“It is very exciting, because you rarely see this kind of therapy studied at all, and it was very rigorously done,” said Dr. Dianne Chambless, a professor of psychiatry at the University of Pennsylvania who was not involved in the study but is now collaborating with the researchers.

Dr. David H. Barlow, a psychiatrist at Boston University, said in an e-mail message that the study was too small to be conclusive but that “the authors should be congratulated for actually taking the first step in doing the hard work of beginning to evaluate treatments” that are widely used without good supportive evidence.

The researchers tested a pared-down version of analysis tailored specifically for panic attacks, the breathless, paralyzing dread that strikes some 1 percent to 2 percent of people, seemingly out of nowhere. Previous studies had found that other kinds of therapy — including exposure techniques, in which people learn to diffuse their anxieties by facing them one small step at a time — can relieve panic attacks in half to two-thirds of patients, depending on the severity and type of anxiety.

In the new experiment, Dr. Barbara L. Milrod, a psychiatrist at Weill Medical College of Cornell University, led a team of therapists who treated 49 men and women with a variety of anxieties. Some were agoraphobic, unable to ride the subway or visit certain parts of town. Others had symptoms of depression or of personality problems, like a disabling dependency on other people or an avoidance of social situations.

Half of the group received a form of relaxation training, in which they learned how to moderate their arousal by tensing and relaxing specific muscle groups. The other half received psychodynamic therapy, working with their therapist in two weekly sessions to understand the underlying meaning of their symptoms — when the reactions first started and how they might be linked to loss, broken relationships or childhood experiences that unconsciously haunted their current lives.

After 12 weeks, 39 percent of those working with relaxation techniques improved significantly on standard measures of anxiety and reported fewer panic-related problems in their relationships and work. But almost three-quarters of those receiving psychodynamic therapy reported similar benefits.

“This is best response rate I’ve seen in a controlled trial for panic,” Dr. Milrod said. “And the therapy was time-limited. I don’t think anyone would care if psychoanalysis cured panic in six years — snore. We wanted to know that what we were doing worked, that it wasn’t malpractice.”

Researchers from Columbia University, the Mount Sinai School of Medicine and Hunter College were also involved.

Studies of this brand of therapy are important for the thousands of therapists around the world who mix and match analytic techniques with other therapies. One former patient treated with this therapy began to have panic attacks after witnessing a young woman die of an illness, said her doctor, Fredric N. Busch, a Cornell psychiatrist and a co-author of the new study.

The patient, who was not a part of the study, described the death as deeply unfair, and in sessions explored perceptions of unfairness in her work and her life, including her childhood. “Once she was able to understand this pattern, the panic became less frightening, she felt safer and was eventually able to get rid of the symptoms,” Dr. Busch said.

The researchers said that even if this approach was not for everyone, it appeared to be especially beneficial for a particular group. In an analysis of individual patient’s responses, the researchers found that those who also had a personality disorder, like avoidant personality, showed significantly greater improvement than those whose symptoms were related solely to anxiety. Patients with multiple diagnoses are usually more difficult to treat.

“This finding was very surprising and there’s absolutely no precedent for it, as far as I know,” Dr. Milrod said.

See also the paper’s abstract:

OBJECTIVE: The purpose of this study was to determine the efficacy of panic-focused psychodynamic psychotherapy relative to applied relaxation training, a credible psychotherapy comparison condition. Des- pite the widespread clinical use of psychodynamic psychotherapies, randomized controlled clinical trials evaluating such psychotherapies for axis I disorders have lagged. To the authors’ knowledge, this is the first efficacy randomized controlled clinical trial of panic-focused psychodynamic psychotherapy, a manualized psychoanalytical psychotherapy for patients with DSM-IV panic disorder.

METHOD: This was a randomized controlled clinical trial of subjects with primary DSM-IV panic disorder. Participants were recruited over 5 years in the New York City metropolitan area. Subjects were 49 adults ages 18–55 with primary DSM-IV panic disorder. All subjects received assigned treatment, panic-focused psychodynamic psychotherapy or applied relaxation training in twice-weekly sessions for 12 weeks. The Panic Disorder Severity Scale, rated by blinded independent evaluators, was the primary outcome measure.

RESULTS: Subjects in panic-focused psychodynamic psychotherapy had significantly greater reduction in severity of panic symptoms. Furthermore, those receiving panic-focused psychodynamic psychotherapy were significantly more likely to respond at treatment termination (73% versus 39%), using the Multicenter Panic Disorder Study response criteria. The secondary outcome, change in psychosocial functioning, mirrored these results.

CONCLUSIONS: Despite the small cohort size of this trial, it has demonstrated preliminary efficacy of panic-focused psychodynamic psychotherapy for panic disorder.

At last my psychodynamic colleagues are engaging in the difficult work of validating our treatments. These authors deserve much praise for the pioneering effort. Its nice, but not critical, that we also like the results!

Add comment February 6th, 2007

The (non)science of interrogations

The debate about psychologist’s participation in interrogations has taken a new turn with the release last week of the new report Educing Information: Interrogation Science and Art — Foundations for the Future. Phase I. Apparently the report was secretly published in December by the Intelligence Science Board but was recently leaked to the Federation of American Scientists, which publicly posted it. [UPDATE May 14, 2007: Charles Morgan claims in the comments that the group preparing this always intended to publish it. I have no reason to doubt him. Evidently I misinterpreted the Nature comment below that the report was leaked.]

It was compiled by a team of Advisers and a Government Experts Committee on Educing Information. Interestingly, these groups include three of the 10 members of the American Psychological Association’s Presidential Task Force on Psychological Ethics and National Security, the so-called PENS Task Force that contained six of nine voting members (there was a non-voting chair) from the military and intelligence communities; half of those six are involved here. The project was directed by Robert Fein, a member of the PENS Task Force.

The report give the Mission

“The Intelligence Science Board was chartered in August 2002 and advises the Office of the Director of National Intelligence and senior Intelligence Community leaders on emerging scientific and technical issues of special importance to the Intelligence Community. The mission of the Board is to provide the Intelligence Community with outside expert advice and unconventional thinking, early notice of advances in science and technology, insight into new applications of existing technology, and special studies that require skills or organizational approaches not resident within the Intelligence Community. The Board also creates linkages between the Intelligence Community and the scientific and technical communities.”

The PENS Task Force, therefore, was not just stacked with military and intelligence officials, but with extremely high-level officials. The choice of these individuals as the people to advise the APA on ethics clearly means that the decision regarding the Task Force’s recommendations was made in advance and members were chosen that would come up with the requisite recommendations. No wonder the Task Force membership was kept secret for as long as possible. There could not possibly be even a hint of legitimacy in a statement by this group that psychologist participation in interrogations was ethical.

It is far past time for the APA to set aside the PENS report, declaring it unacceptable due to the composition of the Task Force and the numerous procedural irregularities that occurred in the preparation of the report.

As for the Educing Information report, it concludes that there is no science-base for interrogations. No expert “knowledge” for the so-called Behavioral Science Consultation Teams (BSCT) of psychologists to use in consulting to interrogators. The BSCT interrogators were just using common sense, and the military’s own SERE (Survival, Evasion, Resistance and Escape) experiences as they strove to figure out how to use detainees’ personal weaknesses to break them down.

The Washington Post reported on the Educing Information report last week in their article Interrogation Research Is Lacking, Report Says.

And the journal Nature today published an article online, which I reproduce here:

Interrogation comes under fire; Tough questioning tactics lack scientific foundation, intelligence agencies told

by Geoff Brumfiel

There is no scientific basis for current interrogation techniques, a US government-funded study has found. The report has stirred up controversy by calling for more research into the matter, angering many psychiatrists who believe such work is unethical.

The 374-page study on “educing information” was conducted by the Intelligence Science Board, an independent panel that advises the government’s intelligence agencies. The report concludes that “virtually none of the interrogation techniques used by US personnel over the past half-century have been subjected to scientific or systematic enquiry or evaluation”.

First published in December, the report became public last week after it was leaked to the Federation of American Scientists, a watchdog group based in Washington DC. Members of the study group declined to comment, citing the sensitive nature of their work.

The report provides a comprehensive review of military and law- enforcement interrogation techniques and finds numerous misperceptions, both within and outside professional circles. For example, it concludes that the belief that torture breaks down a subject’s resistance is without technical merit, as is the effectiveness of strategies such as sleep deprivation. It also finds that professional interrogators have as many erroneous beliefs as novices about how to use body language to spot liars, and concludes that current lie-detection technology is still highly unreliable.

In a controversial final chapter, the report calls for a systematic investigation of interrogation techniques to determine which yield the best information, and suggests reviewing the testimonials of former US prisoners of war to understand whether and how torture worked on them.

Finally, it calls for controlled studies on soldiers undergoing survival training and on college students willing to participate in “benign” research.

Such studies might be useful if they are conducted safely and ethically, says Steven Aftergood of the Federation of American Scientists. He points out that regardless of scientists’ position on the matter, US soldiers and intelligence officers seem to be engaging in harsh interrogation practices in Iraq, Afghanistan and at Guantanamo Bay in Cuba, so they need to know what works, and what doesn’t. “We have not done very well in the absence of research,” he says.

Others disagree. “I doubt very much that any research could be done in a university setting or that any ethical person would do it,” says Alan Stone, a psychiatrist at Harvard University in Cambridge, Massachusetts.

Stone points out that interrogation is often designed to induce stress, and that raises a host of “intractable” ethical issues, such as how to gain consent from study subjects.

The fields of psychology and psychiatry are split over whether to carry out such work. In 2005, the American Psychological Association stated that psychologists could participate in interrogation, but not torture.

The American Psychiatric Association, meanwhile, has condemned any such work by its members. Gregg Bloche, a lawyer and psychiatrist at Georgetown University in Washington DC, says: “This underscores the need to make some rules.”

It should be noted that Educing Information is described on the title page as “Phase 1 Report.” Presumably Phase II will report on research studies on the most effective ways to “educe” information. Does waterboarding work better than threatening to kill family members? How effective is using a fear of spiders to break an individual? Should you allow no more than two hours of sleep a night, or is four ok, as stated in the new Armed Forces Manual on interrogations. Or, just perhaps, the good cop, bad cop routine is as effective as these more experimental techniques.

There is a real danger from “terrorists,” albeit an overblown one. Getting information is important, sometimes. But I, for one, do not trust the “intelligence community,” dirtied by involvement in Guantanamo, Abu Ghraib, Baghram, extraordinary renditions, and the secret CIA prisons, to develop “ethical” and “scientifically-supported ways of educing information. By its nature, there will be no public oversight or control of such research as it will, undoubtedly, be classified. After all, we can’t let our enemies know what to expect when captured. And we certainly wouldn’t want other countries using our taxpayer-funded research to “educe information” from their rebels, or even, heaven forbid, from captured Americans.

Of course, the APA will be delighted to have yet another opportunity to demonstrate the importance of psychology to the “national security” effort. They’ll be hoping that those fat research grants will go to psychologists — and not to their arch-rivals, the psychiatrists — as the profession has demonstrated its loyalty.

The lack of scientific knowledge on how to “educe information” will, no doubt be trotted out by the APA leadership as yet another reason why psychologists must “engage” with the intelligence community by participating in interrogations. This call for “scientific eduction” will likely ignite the next phase of our battle against the use of psychological knowledge for human destruction rather than healing.

8 comments January 24th, 2007

Open Source on Iraq Mortality Study

Boston’s WGBH radio show Open Source with has a show on the Lancet Iraq Mortality Study. It features Les Roberts, Juan Cole, critic Colin Kahl, and several Iraqi bloggers.

Listen here.

Thanks to Tim Lambert for this.

Add comment December 29th, 2006