How to Deceive with Statistics

Someone Else Does My Job Edition

I realize I haven't talked about statistical errors in a while, but nothing readily accessible to the layman has really come up. That is, until this thorough article discussed a recent article on the topic of children and cow's milk:

Mothers who feed their babies cow’s milk in the first 15 days of life may be protecting their children from dangerous allergies later on, says a new study.

"O RLY?" the author of the above article said upon reading the above sentence. After all:

[...] given they claim the exact amount is unknown, making a suggestion to "give a single bottle daily" is highly unusual; not least because it contradicts worldwide recommendations which are based on extensive evidence.

Not given to trusting Prof. Yitzhak Katz of Tel Aviv University’s Department of Pediatrics, Sackler Faculty of Medicine---at least not as a default position---she (I think the author is a she) decided to look up both Professor Katz and the original study.

On Dr. Katz, she found he has received funding from the Israel Dairy Board, which presumably could benefit from adding the 0-6 months demographic to their target audience. In other words, everything he says is probably a lie, right? Well, I don't recommend going as far as the author on this. Partially this is because while the Dairy Board might benefit from an increase in sales to newborns, inducing health concerns and possible allergies later on means they'd lose a lot of revenue to the soy folk, so I'm skeptical that they are really pushing junk science. Follow the Money is good advice, but it not only tells you when someone has an incentive to lie, it also says when they have an incentive to be careful about which lies they tell.

But I also think we should be careful about dismissing Dr. Katz for his Dairy Board connection because this is what the debate kids call an ad hominem attack: namely, we're complaining about Katz's credentials, but not dealing with his argument. In a very real sense, if the science works, the science works, regardless of who paid for it. Follow the Money lets us know when we should be extra perceptive to things like wording, but it doesn't give us license to just dismiss evidence. Fortunately, although the article's author takes a hard line in her rhetoric, her practice is solid in this area. I do so love the magic words "So I decided to dig out the study."

The article is a great example of giving a close reading to the original sources cited in a piece of potentially biased news. It also highlights how a close reading and attention to the wording of claims is important when dealing with statistics. Really, there's not much I can add to this case study. Do read the whole article, please.

The bottom line, given what the study actually says, is I think accurate:

To suggest infants be given a bottle of cow's milk on the basis of this one study, is not only irresponsible, but really quite scary!

In other words, don't do this. The bottom line from the "don't be deceived" angle is this: Be wary of any news item that tries to convince you with the phrase "says a new study." The study might not say that at all; even if it does, it might not lay claim to the level of confidence the article suggests; and even if it does that, the study might not be very well done in the first place!