 |
|
|
L. Neil Smith's THE LIBERTARIAN ENTERPRISE Number 265, April 4, 2004 Boycott Verizon Communications!
Reading Between the Numbers
Special to TLE "According to 'experts', growing numbers of women are traumatised by childbirth and are not capable of child-rearing without professional help." Last week, with these words, sociologist Dr. Ellie Lee cast doubt upon the increasingly popular theory that postnatal depression affects as many as 1 in 5 new mothers. She said the research underlying that claim is "wrong." How can we trust research? The question is nontrivial since studies and statistics form the basis for many of the laws under which we live. If they are wrong, then the laws may be as well. Short of taking a course on statistics and poring over data, the best way to get a sense of which data to trust is through common sense. There are five questions you should demand of any statistic. Who Says So? The purpose of this question is to discover possible bias on the part of those offering the data. The bias may be conscious such as research conducted expressly to win a government grant. The bias may be unconscious such as research conducted by those who have deep ideological convictions that influence the questions asked. Bias does not invalidate findings. Just because a researcher seeks funds or has a personal opinion doesn't mean his finding that 2+2=4 is false. But it does mean you should look at the math more closely. How Does He Know? Imagine a researcher who rang doorbells at random to ask, "Are you a criminal" or "Do you fart often?" That researcher might discover the world to be both crime and fart-free—not because it is but because many people will not admit to either. One of the most common methodological mistakes is to rely upon an unrepresentative sampling, such as polling only Baptists or limiting your sample to 10 people. "Seventy-five percent of American prefer milk to lemonade" is an impressive finding until you realize that only twelve people were sampled, all of whom were Wisconsin dairy farmers. At that point, the surprising statistic is that twenty-five percent preferred lemonade. Demand to know the exact question asked or studied, the size of the sampling, and whether it was random. What does the competition say? Many studies contradict past findings or constitute "surprising revelations." It could be that past or competing studies were flawed; times may have changed. Data, like opinions, can vary and a finding that "eating cheese increases your chance of cancer by 12%" should be considered in light of past or current studies which render different results. It is possible for a multitude of small surveys to be conducted until one of them produces the desired results. For example, after tossing a coin many times, it will land "heads up" nine times in row. From that isolated experience, a researcher could conclude that a tossed penny will come up heads 90% of the time. Do other findings contradict him? What is missing? Does the data tell you enough to evaluate its statements? Consider the statement, "the average salary at this company is $30,000 a year." 90% of employees may make much less than that amount but, when total incomes are divided by total employees, $30,000 may be the "mean" result. A "median" result reflects what the person at the exact middle of the earning range takes home. The "mode" is nothing more than the most frequently encountered figure. Does the figure $30,000 indicate a mode, a median, or a mean? Did Someone Change the Subject? A newscaster states, "reports of domestic violence have increased" and concludes that "domestic violence is on the rise." This conclusion is not justified because the increased reporting may reflect nothing more than a greater willingness on the part of women to contact the police or a greater willingness of police to file the reports. The newscaster has changed the subject from increased reporting to increased incidents. Does It Make Sense? Never allow a statistical finding to override common sense or your own perceptions: guesstimate. That technique involves taking a statistic to its logical conclusion and seeing if it reduces to absurdity. Consider the alarming statement, "over 3,000,000 teenage girls on welfare became pregnant this year." Start with the total population of the U.S.—roughly 300,000,000. Assume that roughly half are male, leaving 150,000,000. Assume a uniform female age-spread of one to seventy-five years, with teenagers (13-19) constituting approximately 9.3%, or 14,000,000. Assume every teenage girl can become pregnant. Divide this figure by the reportedly 3,000,000 pregnant welfare teens and the ratio you get is 4.67. One in five teenage girls is not only on welfare but has also become pregnant in the last year. Does this make sense, does it accord with your own perceptions? The research on postnatal depression may or may not be valid. Lee accuses its advocates of constructing a problem, of "medicalizing motherhood." Lee states, "There is every possibility that...parents will come to experience the normal disruption that parenting brings with it, as highly disabling, and find themselves less able to manage... This risks branding an essential part of life a hazard." Our society rewards those who construct problems. They receive financing and media attention, write books and become "experts." Statistics are tools and those who wield them should be neither glamorized nor ignored. But they should be required to answer basic questions before being included in that rare category: purveyor of truth.
TLE AFFILIATE
Search Amazon.com for ANY BookHelp Support TLE by patronizing our advertisers and affiliates. We cheerfully accept donations!
|
|
|
|
