|Matzoh ball pho from Eatsie Boys, Houston, TX|
This misunderstanding can, of course, be aggravated by poor science reporting. The New York Times recently published an article (authored by Stephanie Strom) reporting the survey of the antibiotic resistant bacteria found supermarket, kosher, organic, or "raised without antibiotics (RWA) chicken in the New York City area. She summarized the results thusly:
Almost twice as many of the kosher chicken samples tested positive for antibiotic-resistant E. coli as did the those from conventionally raised birds. And even the samples from organically raised chickens and those raised without antibiotics did not significantly differ from the conventional ones.The same conclusions are being echoed by various other outlets, from Salon to the Jewish Weekly (some writers focus on the fact that the lead author, Jack Millman, is 17 years old). But I found some problems upon reading of the paper published in F1000 Research (and responsibly linked). First is that the authors reported the resistant microbes as a percentage of isolates - they didn't report it as absolute values. Meaning that unless they had sampled equally among the different categories of chicken, this can be radically different. For example, if there were 100 conventionally raised birds sampled, and 10 kosher birds, but 5 were resistant in both samples, the kosher birds would come up as 50% while the conventional would be 5%. Reporting the total population number is important to interpretation.
But the second sentence will perplex most: after all, "organic" should include an exclusion of antibiotics, while RWA means that the birds could still be raised conventionally just without the use of commercial antibiotics. And yet, the resistance levels weren't significantly different from conventional - how can this be so?
The key could be how the samples were procured. The authors had gathered 213 chicken samples - in order to manage this large set of samples, they decided on a single bacterium per sample. This, alone, should raise alarm bells. But this is how they did it: they took a chicken leg from each sample, dropped in a sterile bag with MacConkey broth, and incubated it overnight. MacConkey broth is a growth medium for bacteria, but most bacteria are actually unculturable - that is, for more than 90% of bacterial species we don't even know how to grow them in controlled laboratory conditions. MacConkey broth is designed specifically to grow lactose fermenting bacteria - this is called a selection regimen. Only certain bacteria will be favored to grow in these conditions. Why? Because the authors wanted to focus on a well studied workhorse: E. coli. And I have a whole different article on why E. coli can be a confusing bug to study.
|Chicken soup, from Pecking Order, Chicago, IL|
Imagine if you're comparing cities. If you took one person from any city in the world, at random, and presumed to judge the characteristics of the population of that city from this one sample, it wouldn't be valid. Now, instead of that one person, you decided to take a descendant from a selected number of residents many generations later. I find it hard to think that it would be representative of the original population.
This, I fear, is the weakest part of this study. The authors technically sampled less than one cell per sample and extrapolated their conclusions to the original. This is not a rigorous study design. But the authors didn't try to oversell their results - the journalists who reporting on this jumped to very broad conclusions, often bringing angst regarding the kosher bird process - I am not even going to discuss the fact that the authors disclosed that their classification system was intentionally skewed to report more antibiotic resistant bacteria than not. Bottom line: there's insufficient information in this study to conclude one way or the other.
I did learn that there's a practice of injecting some antibiotics in eggs prior to the chick hatching, although the paper implied that this is standard practice. In truth, this is "off label" use, meaning that it isn't an approved practice.