650,000 dead given voice in Congress
December 11th, 2006
I just watched the Congressional briefing on the Lancet Iraqi casualty study: “650,000 excess deaths in Iraq”. Speaking were Gilbert Burnham, Les Roberts, and Juan Cole. The briefing was organized by Rep, Kucinich, with the support of rep. Ron Paul.
The briefing was to discuss the October 2006 study Mortality after the 2003 invasion of Iraq: a cross-sectional cluster sample survey, which appeared in the British medical journal The Lancet. This study estimated that 655,000 more Iraqis had died (“excess deaths”) since the invasion than would have died if the prewar rate of death (mortality) had continued. It further estimated that about 600,000 of these had died from violence.
I have great respect for Rep. Kucinich, who kept the entire briefing focused on the effects of the fighting on the Iraqi people and on Iraqi society. It is the first time I’ve seen anyone from the U.S. government focus on Iraqis and what they have experienced and what they are suffering. For example, he asked about the effects on Iraqi society of the loss of so many young men. He also asked about the creation of Iraqi orphans. Unfortunately, none of the three scholars had any real information on these topics, a sign of how little we really know about what is going on in that unfortunate country.
I thought Burham and Roberts did an excellent job of presenting the study. While this was not a methodological seminar, Burham said that they were well aware of the potential for bias and spent months designing the sample design so as to include all households. This declaration constitutes an explicit statement that the so-called “Main Street Bias” proposed by British scientists is not present to any meaningful degree. One may question the honesty of the Lancet study authors, but speculation about a massive MSB after such a definitive statement does require questioning their integrity.
Juan Cole, of the Informed Comment blog, presented various evidence from the media and other sources that supported the Lancet authors’ position that the vast majority of deaths are not presented in the press, making the results of this study less surprising. Roberts pointed to the statements in the new Iraq Study Group report that the US military had radically underreported the extent of violence in Iraq. In particular, as the Associated Press put it, the report stated:
“The panel pointed to one day last July when U.S. officials reported 93 attacks or significant acts of violence. ‘Yet a careful review of the reports for that single day brought to light 1,100 acts of violence,’ it said.”
If, on average, one Iraqi died in each such attack, the mortality rate would be greater than that in the Lancet study. Thus, the reported rates are not implausible, as many critics claim.
Of course, the fact that the mortality rates are not implausible does not mean that they are therefore correct. While many epidemiologists and others have defended the study, some experts in this area, most notably the eminent Norwegian researcher Jon Pedersen have criticized the study. Like all studies on important matters, this one does deserve careful scrutiny. But it does not deserve to be dismissed by the press in a way that similar studies with results more comfortable to the United States government are not dismissed. The existence of “controversy” should not be an excuse to ignore that, as a consequence of U.S. government action, hundreds of thousands of Iraqis have needlessly died.
Les Roberts again made the point that their data implies that the majority of deaths in Iraq are from violence, whereas alternative accounts from Iraq Body Count, the Brookings Institution, or the Iraq Ministry of Health imply that only a small percentage, perhaps 10%, of deaths in Iraq are from violence. He again, as he has done since the study came out in early October, has called upon the press to visit graveyards and ask if the majority of deaths is from nonviolent or violent causes. Roberts again called, as these authors did after their 2004 study, for another research group to investigate the Iraqi mortality rate and confirm or invalidate the Lancet study. It is disturbing that, in the two years between theses researchers’ 2004 and 2006 studies, no other group did attempt such a replication. Given the numbers of surveys conducted in Iraq on other controversial issues, such as attitudes toward attacks on Coalition troops, it should he relatively easy for this study to be replicated. Perhaps all of us, whatever our evaluation of this study, can echo these calls to the press and to other survey researchers.
Movingly, Rep. Kucinich ended the briefing by emphasizing “the imperative of human unity” “that we recognize the imperative of human security,… that each of us has a right to survive.” And: “It is an imperative to focus on the imperative of peace. War is not inevitable.”
Kucinich seems among the very few in the public arena who realize what is truly at stake for the human race in an era of modern technology. Given the nature of this technology of warfare, either Rep. Kunich’s call will be heeded or one of the continual conflicts will spark an all out war that will destroy us all.
[Cross-posted on Daily Kos and OpEdNews.]
Entry Filed under: Electoral Politics, Iraq, Middle East, Mortality, Politics, Public Health, Research Methods, Science, Social Issues, War and Peace
21 Comments
1. Donald Johnson | December 11th, 2006 at 4:04 pm
Good post. Translation–I agree with your viewpoint. I’m not certain the casualty range in the Lancet 2 paper is correct, but it deserves much more press attention than it got. Or perhaps more the point, the issue of how to determine the violent death rate in Iraq deserves much more attention. People in the press, for the most part, don’t seem to be thinking about this very much–what’s to stop them from asking why other surveys haven’t been done, or from trying to do what Les Roberts has suggested by going to visit graveyards?
2. admin | December 11th, 2006 at 4:29 pm
Yes, Donald.. Despite all the debate about methodology, Roberts makes the very good point that the basic assumption can be tested by reporters, should they care to. We should put pressure on the press to do so.
3. Hal O'Leary | December 12th, 2006 at 12:08 am
KUCINICH FOR PRESIDENT. Of our 535 reresentives in congress it is rather distressing to find only one with the courage to tell it like it is. Even if we Americans
are egocentric enough to think that only American lives are worthy of consideration,
why are we not out in the streets protesting the NEEDLESS slaughter of more than 3000 of our young men and women. Americans, have you no shame. There is only one answer to this atrocity and that is the IMPEACHMENT of this administration. Please America, WAKE UP BEFORE ITS TOO LATE.
4. Ral173 | December 12th, 2006 at 3:12 am
The weakness of the study may be the data collectors. I believe that the report said that they were Iraqi academics, who, if so inclined would understand how to skew the results.
5. Robert Shone | December 12th, 2006 at 5:14 am
Stephen Soldz wrote:
>> One may question the honesty of the Lancet study authors,
>> but speculation about a massive MSB after such a definitive
>> statement does require questioning their integrity.
Or, alternatively, one may disregard all speculations arising from attribution of motive (“dishonesty”, “integrity”, etc) and instead concentrate on matters such as logic, supporting evidence, etc.
The Lancet authors have asserted that “main street bias” doesn’t affect their study – they have stated that they included “all” streets in the sampling process. But in order to include “all” streets they would have to conduct their study in a fundamentally different way than that described in their published account. Nowhere have they described the methdology they used to include “all” streets. We’re left merely with their assertion that they did include all streets.
In such an important study, this is deeply unsatisfactory. The scientific community is unable to properly assess how they avoided this bias. They’re expected instead to simply accept the word of the Lancet authors that they “made efforts” to avoid it. The MSB authors are correct to say that the burden is on the Lancet team to demonstrate how they avoided such a bias.
In raising these matters, it isn’t necessary to “question the integrity” of the Lancet authors. (It’s simply a call for the Lancet authors to provide more than assertions on the question of including “all” streets). The MSB authors nowhere imply that the Lancet authors are dishonest, and neither do I. Science is about supporting one’s claims/assertions. It’s not about expecting people to accept your claims/assertions based on an attribution of honesty/integrity alone. That would be absurd.
6. Eric Brunner-Williams | December 12th, 2006 at 11:02 am
(via Cole)
Thanks for blogging the hearing. I was a DK delegate to the Maine Convention, and I’m not surprised that Dennis is doing the right thing now, following up on the Lancet Study, and setting up the next Defense Appropriations fight.
We’ll won’t win the next DA vote, we’ve never won any prior, and probably won’t until we counter “support the troops” with something more compelling in domestic political calculus like “save the army”.
7. Nelson Wight | December 12th, 2006 at 11:24 am
The indifference of our hegemonic-oriented representation in Washington is beyond understanding.
How good it is to see at least Congressmen such as Ron Paul (Libertarian-R-TX) and Dennis Kucinich (Progressive-D-OH) stand up and present this study to those IgnorANUSes who say they are representing America and Americans.
There are not enough words in our language to commend them fully, nor enough uncharitable words to condemn those others who demonstrate ignorance, greed and lack of courage.
Thanks to Juan Cole and all.
8. George | December 12th, 2006 at 2:33 pm
The tragic loss of innocent life is another example of the evil that has been thrust on the Iraqis. Chavez was absolutely right when he likened Bush to Satan. Only Satan could have caused what we see in Iraq now.
9. John P | December 12th, 2006 at 2:46 pm
I think the main street bias argument is just as valid as the phase of the moon argument. I don’t think the phase of the moon affected the results.
Science is not just about supporting one’s claims/assertions. It is also about testability. The only way to truly test Lancet 2 is to redo the study and see what the result is. Arguing back and forth about main street bias, or significant figures, or anything affecting the study will never resolve the question. Someone with the the resources and the courage to redo the study will. I applaud Burnham and Roberts for their effort and honesty.
10. Tony Gerard | December 12th, 2006 at 2:57 pm
The first rule of statistical analysis is to look at the results and ask if they reflect common sense. Sometimes this is referred to as a question of “external validity.”No matter how apparently valid the methodology, the results can do a poor job of mirroring the real world. The usual number given for the number of Iraqis dying daily is 100; in a year, that would add up to 36,500. We have been in Iraq for about three and one half years, giving a total of approximately 150,000 civilian casualties, at the current rate of killing. That leaves a difference of half a million fataltities for which the Lancet Report must account. That difference is the first issue the study authors must address.
11. Just Foreign Policy News &hellip | December 12th, 2006 at 4:05 pm
[...] 2) 650,000 dead given voice in Congress Stephen Soldz, Psyche, Science, and Society, December 11th, 2006 http://psychoanalystsopposewar.org/blog/2006/12/11/650000-dead-given-voice-in-congress/ I just watched the Congressional briefing on the Lancet Iraqi casualty study: “650,000 excess deaths in Iraq”. Speaking were Gilbert Burnham, Les Roberts, and Juan Cole. The briefing was organized by Rep. Kucinich, with the support of Rep. Ron Paul. The briefing was to discuss the October 2006 study, ” Mortality after the 2003 invasion of Iraq: a cross-sectional cluster sample survey,” which appeared in the British medical journal The Lancet. This study estimated that 655,000 more Iraqis had died (”excess deaths”) since the invasion than would have died if the prewar rate of death (mortality) had continued. It further estimated that about 600,000 of these had died from violence. [...]
12. LuAnne Hightower | December 12th, 2006 at 5:22 pm
This study, as I understand it, was conducted by Johns Hopkins University. I believe that the corporate media have chosen to give it the label of “Lancet study” in order to give it less weight, since the general populus recognizes Johns Hopkins and is mostly unfamiliar with the Lancet journal. I’d bet dollars to doughnuts that if it were a positive statement on the war, it would be touted far and wideas the “Johns Hopkins Study.” Maybe the bloggosphere can straighten out the spin a bit…
13. admin | December 12th, 2006 at 5:58 pm
LuAnne,
You point to something that has bothered me: why is it referred to as the Lancet study. after all, all the Lancet did was publish it. You’re right that it should be referred to as the Johns Hopkins study.
14. Donald Johnson | December 12th, 2006 at 10:42 pm
Tony, you’re missing the point. The Lancet (Johns Hopkins) authors are claiming that the press and official sources miss most of the deaths. That happens in some wars and not in others. The question is which group the Iraq War would fall in.
What we need is less squabbling about this study and more calls for more studies. If it’s possible to do polls in Iraq, and it has been, then it should be possible to settle this issue if people in charge really cared. They clearly don’t.
Though that said, what we need even more is a recognition that whatever the true number, the war has been a disaster.
15. Robert Shone | December 13th, 2006 at 6:20 am
John P wrote:
>> I think the main street bias argument is just as valid as
>> the phase of the moon argument. I don’t think th
>> phase of the moon affected the results.
In that case, John, you disagree with Gilbert Burnham (the study’s lead author). Burnham stated that his team made efforts to “to reduce the selection bias that more busy streets would have” (indicating that he accepts there is such a bias to be avoided).
John P wrote:
>> The only way to truly test Lancet 2 is to redo the
>> study and see what the result is.
If you redo the study you may simply reintroduce existing biases (resulting from importing epidemiological methods into conflict zones). The point about the main street bias research (etc) is to discover such biases and in so doing to improve the application of these epidemiological surveys to measuring violent deaths in conflicts (an application that’s not well-validated, as Burnham et al acknowledge).
Unfortunately, conflict mortality is unlikely to end with Iraq. There will undoubtedly (and tragically) be other studies into deaths from future wars. If people are concerned with accuracy (and I think they should be), they might want to seriously consider how these studies can be improved. Identifying biases is a good start. Gilbert Burnham stated in an email to me that because so few surveys of this type have been conducted in conflict zones, and particularly in urban, middle development level countries, there is no standard or norm for them.
I think multidisciplinary studies (including those of the main street bias type) are urgently needed in this field. And they should involve statistical methodologists from other areas – epidemiologists have no monopoly on conflict mortality. I’m concerned that many people are simply dismissing research such as main street bias out of hand, as it implies criticism of the Lancet study (which to some is blasphemy). I think this is unfortunate – the anti-war case doesn’t rely on simply going with the highest death estimate available.
LuAnne Hightower wrote:
>> I believe that the corporate media have chosen to give
>> it the label of “Lancet study” in order to give it less weight
I’m not convinced by this. In the UK, the Lancet is fairly well-known as a prestigious journal, so the term “Lancet study” I think has an authoritative, respectable, scientific “ring” to it. On the other hand (in the UK) “John Hopkins” isn’t a well-known academic establishment.
16. Robert Shone | December 13th, 2006 at 9:16 am
Donald Johnson wrote:
>> The Lancet (Johns Hopkins) authors are claiming that the press and
>> official sources miss most of the deaths. That happens in some wars and not
>> in others. The question is which group the Iraq War would fall in.
Donald, this is a false dichotomy. It’s not an either/or question, but a question of the degree to which media and official sources miss deaths. For example, did they miss 51% of deaths or 90%. Both amount to “most”, but in the former case the Lancet study is way, way off.
For example the prestigious scientific journal, Nature, asked Jon Pedersen about this:
He [Pedersen] says that violence has become more frequent since his study, but doubts whether the real number can be so much bigger than media reports suggest. [...] “We are told about at least 30 to 40 deaths per day just from news reports,” says Pedersen. “But 500 per day is very different.”
(Nature, 11 October 2006)
I think Tony Gerard’s point is valid (and it’s echoed by Pedersen, IBC, the MSB team and others): given the Lancet figure is higher by an order of magnitude than any other count/estimate (implied not just by media or official sources, but figures suggested by the likes of Pedersen), then “the purveyors of this outlier must make a good-faith effort to explain why all the other information is so badly wrong” (to quote Michael Spagat).
Simply asserting that “most” deaths are missed by media and official sources doesn’t really do it. It’s drifting away from science and into the realm of unquantified assertion (of the type you’d more typically find in, say, media criticism polemics).
17. Robert Chung | December 13th, 2006 at 11:13 am
Robert Shone wrote:
Can’t disagree with this, since demography is a multidisciplinary field — a multidisciplinary field that happens to focus on these sorts of issues.
I’m sure that you are correct, and that many people are simply dismissing it out of hand– but I don’t believe many professionals do. I do, however, believe that for this particular situation many professionals dismiss an overall bias factor of 3.
18. Donald Johnson | December 13th, 2006 at 2:46 pm
Robert–
There are some wars where deaths are underreported by factors of 5 or 10, and others where coverage is much more complete. So again, the question is which sort of war do we have in Iraq? I could believe it’s only a factor of 2 or less undercoverage, or maybe it’s much higher–there’s some anecdotal evidence (from Iraqi bloggers, for instance) that the violent mortality rate is much higher than IBC’s measured rate of 1 in 500.
But I don’t pretend to know. My personal leanings vary somewhat from day to day, but I would have guessed something like 150-200,000 violent deaths before Lancet 2. But my personal CI goes from what IBC could stomach all the way up to what Lancet 2 claims is possible.
19. Global Strategy and Imple&hellip | December 15th, 2006 at 12:07 pm
[...] Stephen Soldz blogs the Paul-Kucinich hearings on civilian casualties in Iraq, at which I testified. [...]
20. Robert Shone | December 16th, 2006 at 5:41 pm
Donald Johnson wrote:
>> There are some wars where deaths are underreported by factors of 5 or 10
Remember that the “5-10″ factor isn’t an established certainty. It’s based on accepting that the studies providing those high estimates, like L2, are correct. (It’s like saying we “know” that the factor is greater than 10 in Iraq, because of L2. The point is we don’t “know” – it’s circular).
Donald Johnson wrote:
>> there’s some anecdotal evidence (from Iraqi bloggers, for instance) tha
>> the violent mortality rate is much higher than IBC’s measured rate of 1 in 500.
There’s also good anecdotal evidence that the violent mortality rate is far lower than L2’s and more in line with the IBC/Pedersen view. For example: http://www.dailykos.com/story/2006/10/11/135644/20
21. Robert Chung | December 19th, 2006 at 1:27 pm
Donald and Robert (and Tony):
Anecdotal evidence is quite useful for developing hypotheses but one should always be careful about extrapolating from anecdotes, and you’d need a lot of anecdotes to trump an actual survey. ‘Til now, there are only two actual surveys that collected the data needed in order to compare pre- and post-invasion mortality rates.
Robert Shone wrote:
Yeah, I read that when it came out. He misunderstood the role that sampling fractions play in estimation, and also assumed that larger governates are more violent. I don’t know whether that was so for the 2006 Burnham study but it doesn’t appear to have been so for the 2004 Roberts study.