Join Torture Corp!
Education for the furure! Get it now:
Add comment October 20th, 2006
Archive for October 20th, 2006
More evidence that Iraqi mortality figures are bogus
According to this report, Le Monde is reporting that the Iraqi government has ordered the Ministry of Health to not release further figures:
Iraq ‘hiding true casualty figures’
THE Iraqi Government has told medical authorities not to reveal to the UN the true extent of civilian casualties in the country’s conflict, French newspaper Le Monde said today.
The daily quoted a telegram sent by the head of the UN mission in Iraq, Ashraf Qazi, to headquarters in New York, in which he said: “This development risks damaging the capacity of the UN’s Assistance Mission to report the number of civilians killed or injured.”
Since July 2005 the UN has used data provided by Baghdad’s Forensic Institute and the Iraqi health ministry to form an estimate. The estimate “was certainly imperfect but an indicator nonetheless of the growing number of civilian victims”, the telegram said.The latest report said that 3590 civilians died a violent death in July and 3009 in August, figures which it said were “unprecedented”.
But the telegram quoted by Le Monde said that on September 21, one day after publication of the report, Iraqi Prime Minister Nuri al-Maliki wrote to the health ministry with instructions not to disclose more figures.
In an investigation published in British medical weekly The Lancet earlier this month, US and Iraq specialists estimated that more than 600,000 civilians died a violent death between March 2003 and July 2006.
Of course, this wouldn’t automatically invalidate earlier figures. But reports have frequently referredd to political manipulation of these figures, as in this May 25, 2006 article by the Christian Science Monitor’s dan Murphy:
What until late 2005 used to be a routine reporting visit to the morgue - the best way to figure out the level of violent death in Baghdad - has now become a minefield of frustration and implied threat.
After a person picks up a permission letter from the bureaucrat at the ministry public affairs office, he presents it to the morgue director’s secretary. But now, she immediately directs the person to go to the compound’s security office. “The director is in, but you have to talk to Major Kassem first.” Why? “You just have to.”
Going back outside, Major Kassem is tracked down inside a small air-conditioned trailer. He has a broad smile, and no uniform or badge.
Asked what his role is he explains: “I work for the Interior Ministry, but I’ve been assigned here to help coordinate the FPS.”He then explains that there’s no need to speak to ministry officials, that he will provide all available information. He gives the monthly murder totals from the beginning of the year, though UN officials and Sunni Arab politicians say the ministry has taken to under-reporting the numbers, under pressure from Shiite militias.”
I, further, see no reason to believe that, under these conditions, the correct nation-wide data is even being carefully collected and sent into Baghdad. As one who has worked extensively with these government datasets in this country [we call them "administrative data sets"], I know how inaccurate and incomplete they can be, even under the best circumstances. Collecting and transmitting data is usually the last priority in human service settings. In times of stress, such as civil war, it is virtually unimaginable to believe that these data systems would work. Thus, I don’t find a large discrepancy between the Iraq Body Count numbers and other mortality figures to be at all surprising.
This applies to all the data systems in Iraq. Without extensive evidence that they are in fact working decently, I see no reason to believe that they are. Thus, discrepancies between survey-based mortality figures and official morgue and hospital figures are not only not surprising, but to be expected. This is a large part of why I think the IBC critique of the Lancet study is largely bunk. Its arguments rely to a gerat degree on the ptoper functioning of the countries information infrastructure. When electricity is down to under 3 hours a day in Baghdad, information systems are certainly not a priority in the best of times. These are near the worst of times.
If IBC has good, credible evidence that these information systems are functioning, they should present it. Otherwise, most of thie arguments can be reasonably interpreted as evidence that these systems are not, in fact functioning.
Of course, the weakness of IBC’s critique does not mean that the Lancet mortality study is necessarily correct. I’m still waiting for the main street bias issue to be sorted out. that seems to be to be a potentially greater threat to the validity. [Please note, I said "potentially greater threat to the validity." I am not endorsing this critique, only saying it should be sorted out.]
Add comment October 20th, 2006
Possible flaw in Lancet study worth considering
Here is a note I posted on the Media Lens Message Board in response to heated controversy around the claims of several British scientists to have found a flaw in the sampling methodology in the new Lancet Iraq mortality study [I've posted the piece describing those scientists' ideas below my comments]:
Folks,
I’m not known as a defender of IBC. And I think their recent citique of the Lancet study is weak. But I think the issue raised here of “main street bias” is a serious one and should be taken seriously. I had not wanted to write anything till I could write a comprehensive article, but am disturbed by the tone here.
IBC is right in (at least) one respect. Studies that result in surprising findings should be subject to careful scrutiny. Of course, it’s especially difficult in the present situation where major policy implications flow from one’s critique. It’s also difficult in the gotcha environment where Josh D. jumps on every statement to essentially accuse Lancet defenders of bad faith and its authors of fraud while those who support the study jump on IBC.
While I personally find certain things that IBC has done to be despicable [not making a much greater effort to correct media misreporting of their numbers; likening critics to terrorists; the tone of the Lancet critique), that is irerlevant to the question of the quality of the Lancet study.
The Lancet numbers are surprising. I think the study is of high quality. Yet, that doesn't mean one shouldn't look for flaws. Most of the one's mentioned, such as too few clusters are silly. As many noted, the number of clusters affects the size of the confidence intervals. Also, the relatively low design effect (1.6) suggests this isn't a problem. Of course more clusters are better. But, when designing and conducting a survey, one always balances costs and benefits. The balance in the study is reasonable.
But the possibility of a subtle bias in the sampling design is the crucial issue. If the sample is unbiased, it is hard to come up with an explanation of how such a mortality rate could arise without fraud, which I doubt. [I thought carefully about this last night and think there's enough evidence in the study to rule out [CORERCTED 10-20-2006 1:23EDT] explicit fraud by interviewers at all likely.]
I have seen speculation about some bias introduced regarding these procedures. I’m not at all sure that these authors are correct. But I do think their point needs to be answered. If the sampling strongly over-included main streets, I could imagine it biasing the study.
Unfortunately, I will be out of town for the weekend and away from computer. But I did want to weigh in and say “Cool it folks.” I’m an antiwar activist, but I’m also a scientist. If the study is right, I want to know it. But, if its wrong, I also want to know that.
I also agree with IBC that the number of casualties in Iraq is too high, whether 100,000 [IBC numbers times 2 for nonreporting bias that they admit, andJon Pederson's: estimate] ] or 655,000. I believe the magnitude makes a difference. But I also want to find the truth, whatever it is. I hope others will join in that.
****** Lancet Critique *****
Lancet study fundamentally flawed: death toll too high
October 19, 2006 – 1 page – For immediate release:
Researchers at Oxford University and Royal Holloway, University of London have
found serious flaws in the survey of Iraqi deaths published last week in the Lancet.
Sean Gourley and Professor Neil Johnson of the physics department at Oxford University and Professor Michael Spagat of the economics department of Royal Holloway, University of London contend that the study’s methodology is fundamentally flawed and will result in an over-estimation of the death toll in Iraq.
• The study suffers from “main street bias” by only surveying houses that are
located on cross streets next to main roads or on the main road itself. However
many Iraqi households do not satisfy this strict criterion and had no chance of
being surveyed.
• Main street bias inflates casualty estimates since conflict events such as car
bombs, drive-by shootings artillery strikes on insurgent positions, and market
place explosions gravitate toward the same neighborhood types that the
researchers surveyed.
• This obvious selection bias would not matter if you were conducting a simple
survey on immunisation rates for which the methodology was designed.
• In short, the closer you are to a main road, the more likely you are to die in violent activity. So if researchers only count people living close to a main road then it comes as no surprise they will over count the dead.
During email discussions between the Oxford-Royal Holloway team and the Johns
Hopkins team conducted through a reporter for Science, for an article to be published October 20, it became clear that the authors of the study had not implemented a clear, well-defined and justifiable methodology. The Oxford-Royal Holloway team therefore believes that the scientific community should now re-analyze this study in depth.
The team can be reached for comment at;
Gourley: s.gourley1@physics.ox.ac.uk
Johnson: n.johnson@physics.ox.ac.uk
Spagat: M.Spagat@rhul.ac.uk
1 comment October 20th, 2006