Over the years I've been involved in a fair amount of cluster sampling such as that used in the study by the Johns Hopkins Bloomberg School of Public Health, and the resulting mortality numbers seem pretty solid to me, if mind blowing.
For me, keys to the accuracy of the numbers are:
The pro-active nature of the sampling. Virtually all previous surveys on civilian deaths have been passive; that is, statistics were gleaned from secondary sources. With the Hopkins survey, the numbers were compiled in the field through interviews with the families of the victims.Most surveys on Iraqi civilian war deaths have been passive in the extreme. This is because most have been based on news media reports. The consensus conclusion of these surveys has been that between 40,000 and 50,000 Iraqis have died since the U.S.-led invasion in March 2003.
Who did the sampling. Four-member teams of medical doctors fluent in English and Arabic led by supervisors did the interviews. Death certificates and other documents were examined to determine exact cause of death.
How the numbers were crunched. The statistics were derived from 50 population clusters determined by size of area. Heavily populated Baghdad had 12 clusters while Kerbala and four other sparsely populated areas had one each.
This survey method has two big problems: