Drug contamination in milk?

Or how to read statistics.

Goat milk ice dessert, Dosi Restaurant, Houston, TX

On NPR’s The Salt blog, Dan Charles painted an ominous tone about antibiotics in milk. 

“…a new report from the Food and Drug Administration reveals that a few farmers are slipping through a hole in this enforcement net.”

Appropriately and responsibly, he provides a link to the FDA report, “MILK DRUG RESIDUE SAMPLING SURVEY”. Reading the report itself, however, one comes away with a more optimistic tone:

“ the small number of positives in both the targeted and non-targeted groups is encouraging and the FDA continues to be confident in the safety of the U.S. milk supply”

So, what is going on? 

Let's dissect what's in the report.  What is the difference between “targeted” and “non-targeted”? The targeted group are farms known to have already violated drug residue tests in tissues from culled dairy cows at slaughter. The non-targeted group are controls - farms that don’t have violations against them. Milk from each group are tested for the presence of any one of 31 antibiotics - but even these tests aren’t all equal. For the ones with agreed safety levels, the test registers a violation if it exceeds that level - which is measured in parts per billion (ppb). For example, bacitracin has a tolerance limit of 500 ppb, while ampicillin has a tolerance limit of 10 ppb: these tests are highly sensitive. For antibiotics with insufficient information about their tolerance limits, just being able to identify them is sufficient to trigger a violation: that is, the tolerance limit is set at 0 ppb. Potential false positives should be expected for tests this sensitive. 

But the survey was good about population sizes - 953 farms were in the targeted group, 959 farms in the controls. And even among the targeted groups, only 1 sample had more than about 30 ppm in gentamicin. The 1% figure used by Charles is likely  by computing the 12 positive results/953 samples in the target group. 

This is a misleading use of the data, because it is really a matrix of 953 x 31 tests - or actually, 12/29543 = 0.04%. And most of those were in those 0 ppb tolerance drugs, the majority of which are are single positives for drugs set at 0 bpp acceptable limits. The main one was for the drug forfenicol, which turned out 6 positives. Guess what, 4 samples in the control groups also turned up positive for florfenicol. For numbers this small, they are really insignificant in a statistical sense. These could very well be just noise in the system. 

But the rest of the story tries to imply intentional use of banned antibiotics in the dairy industry - a desperate (IMHO) attempt at drumming up drama over data that could just be noise, experimental error that is normal in the course of large sampling surveys.