No. Snopes has not been Snoped. Note the date of the

Quote:

‘First Ever’ Study Comparing Vaccinated and Unvaccinated Children Shows Harm from Vaccines?

This study, with its suspect statistics and devil-may-care attitude toward methodological design, is a case-study in how to publish a misleading paper with faulty data

By
Alex Kasprak
May 20th, 2017



In April 2017, anti-vaccine groups seemed to have finally gotten what amounted to the Holy Grail for their cause: an allegedly large-scale, peer-reviewed study showing the links between vaccines and autism among a large population of children.

Vaccine skeptic groups, who reject the wide body of scientific literature refuting that link between vaccines and autism, have long sought such a study, but they’ve been hampered by practical concerns, most notably the ethical implications of withholding vaccines from a large group of children.

Released to heavy promotional fanfare on anti-vaccine websites and social media, a 24 April 2017 study published by the Journal of Translational Science claimed to be that Grail.

The study (titled “Pilot Comparative Study on the Health of Vaccinated and Unvaccinated 6- to 12-Year Old U.S. Children”) neatly solved the problem of withholding vaccines by surveying parents who had already chosen not to vaccinate their children.

Using an online survey of 415 mothers of homeschooled children, the study concluded that vaccines can increase the risk of neurological developmental disorders, particularly in cases of preterm birth.

The anti-vaccine website Age of Autism, which also helped raise money for the study, reported its findings in glowing terms:

As parents have long expected, the rate of autism is significantly higher in the vaccinated group, a finding that could shake vaccine safety claims just as the first president who has ever stated a belief in a link between vaccines and autism has taken office.

The only problem? The paper is a identical version of a paper briefly published in Frontiers in Public Health in 2016 before being disowned by the publisher.

This Translational Science version, as well, was pulled from the website with some reports that it had been retracted as well. As of 18 May 2017, however, the study reappeared on that journal’s website with no public comment for why it was removed or returned.

As we will describe below, these de facto retractions and high level of scrutiny stem not from a conspiracy to silence work critical of the medical establishment, but from the myriad ethical, methodological, and quantitative problems inherent in the study and to the research group behind it.

The Researchers, Their Funding, and the “Peer-Reviewed” Journals

Though hailed by some blog posts as a truly “independent” research project, the lead author of the study, Anthony Mawson, is far from an impartial player in this debate: he is a vocal supporter of Andrew Wakefield, a controversial doctor who is arguably the the father of the anti-vaccine movement.

Wakefield first proposed the connection between vaccination and autism in a 12-child case study built with data that was misrepresented and collected without ethical approval, a work that has since been retracted.

At the time of the study, Wakefield was working with a lawyer to create a class-action lawsuit against makers of the measles, mumps, and rubella (MMR) vaccine. Wakefield had also filed a patent for a replacement MMR vaccine that he hoped to develop.

Mawson, who signed a petition called “We Support Andrew Wakefield,” alleged in a 2011 lawsuit that he lost an academic post due to his views on vaccine safety. (That lawsuit was dismissed.)


Mawson’s vaccine study was funded by two anti-vaccine groups: Generation Rescue, founded by anti-vaccine activist Jenny McCarthy, and the Children’s Medical Safety Research Institute, founded by vaccine skeptic Claire Dwoskin.

Web sites such as Age of Autism ran ads calling for donations to Generation Rescue, containing an explicit statement that the money would go toward funding the study.

The journals that published the study are just as a problematic as the donors, though for different reasons: both journals have been accused of predatory practices. These types of journals profit from academia’s relentless focus on publication by charging large publishing fees in lieu of editorial oversight. Translational Science charged Mawson et al $2,000 to publish their study.

Additionally, both journals’ commitment to the peer-review process is questionable at best. For the study’s 2016 incarnation, Frontiers in Public Health asked Linda Mullin Elkins, a chiropractor with no published research to her name on the subject of autism, or any background in vaccine research or epidemiology, to review the study.

This same journal, just four months prior, retracted a paper about “chemtrails”, a popular topic in conspiracy circles alleging government-hidden harm from chemical trails made by planes in the sky.

While there is less of a publication history by which to judge the Journal of Translational Science — its first issue was released in July 2015 — it is not widely cited and is not indexed on the National Institute of Health’s MEDLINE database, a repository for abstracts of medically relevant research.

Problems With Their Data Collection

But even if the funding sources and authors’ histories show bias, they still could have designed a careful study with high-quality research, right? In theory, yes. But in practice, the study design was riddled with flaws.

On its face, this was a fairly simple survey study in which the authors, through a partnership with a homeschooling network, distributed an online survey link to parents asking a series of demographic and medical history questions about their children, as the study’s authors describe:

A number of homeschool mothers volunteered to assist [the National Home Education Research Institute, NHERI] promote the study to their wide circles of homeschool contacts. A number of nationwide organizations also agreed to promote the study in the designated states. The online survey remained open for three months in the summer of 2012. Financial incentives to complete the survey were neither available nor offered.

Right off the bat, the pool of participants in the study differs demographically from the U.S. population as a whole. Parents who homeschool are less likely to view vaccines as safe than parents of children in public schools.

Homeschooled children, as a whole, are less likely to make annual trips to the doctor (and, based on the study’s own data, those whose parents both homeschool and don’t vaccinate have even fewer doctor visits).

Those infrequent trips to the doctor make parents less likely to report medical issues in an online survey — because they may simply not know that the issues are there. Vaccinated children are more likely to be diagnosed with health conditions, but that does not mean that they necessarily have more health issues.

The authors of the study acknowledged that their sample was not intended to be representative of the broader U.S. population:

The object of our pilot study was not to obtain a representative sample of homeschool children but a convenience sample of unvaccinated children of sufficient size to test for significant differences in outcomes between the groups.

But even if the study’s authors weren’t aiming to build a representative pool of participants, they should at least have built in a measure to make sure no further bias was introduced based on those participants’ medical views.

This would have been crucial in light of the fact that the nature, and desired finding of the study was prominently advertised and explained on multiple anti-vaccine websites.

The study’s authors also made no effort to track who received the survey or what percentage of people from different demographic or ideological groups ignored calls to participate (a research metric known as the response rate). This raises the possibility that people who wanted to see a study on vaccine safety were more likely to respond to the survey:


Initial contacts were made in June 2012. NHERI contacted the leaders of each statewide organization by email to request their support. A second email was then sent, explaining the study purpose and background, which the leaders were asked to forward to their members.

A link was provided to an online questionnaire in which no personally identifying information was requested. With funding limited to twelve months, we sought to obtain as many responses as possible, contacting families only indirectly through homeschool organizations.

In short, the study’s design was flawed from the start.

Problems With The Study’s Statistical Analyses

The authors then dressed up their flawed, potentially biased data set in the cheap Halloween costume of a statistically responsible study.

No amount of statistics, however, will get around the fact that there were not enough people in the study to address the questions they claim to have investigated.

Only 47 children in the 666-person study had what the authors defined as a neurological developmental disorders (also called an NDD), an already broadly-defined category that includes multiple conditions. Despite this, viral blog posts reporting on this study repeatedly state that autism risk, specifically, was higher in the vaccinated population.

What those headlines fail to convey is that such a claim rests entirely on the nine people in the study who were actually diagnosed with autism.

The grossly insufficient sample size is most apparent in the lack of precision in the final results. The authors use something called an “odds ratio” to present their analysis — a measure that compares the relative odds of two populations with differing medical histories (i.e. vaccinated versus unvaccinated) developing a certain medical condition (i.e. NDDs). When scientists use odds ratios, they give a measure of how precise the odds are — a range of uncertainty that can be thought of as, essentially, as a margin of error.

In the Mawson study, they claim (for example) that a preterm birth combined with a vaccination caused a 6.6-fold increase in the odds of a neurological condition or learning disability.

This result, a remarkably large odds ratio, comes with a laughable amount of precision — a range from 2.8 to 15.5. That wide a range of possibility is a big red flag, and is representative of many of the associations documented in the study.

The reason there is a range of possible outcomes is because odds ratios are generally pushed through statistical models meant to untangle the competing influences of other factors in their data, isolating the specific effect of one risk factor on an outcome.

This is a crucial step for Mawson et al, because a significant number of children included in the NDD pool were also born prematurely, which multiple studies have already robustly linked to NDDs.

The competing influence of preterm birth on NDDs is arguably the largest methodological hurdle for the researchers to get around. The authors needed to make crystal clear that the association between NDDs and vaccination holds without the influence of preterm birth, or else the study is meaningless. They claim to have done so, but provide next to no documentation on the specifics of that process.

Without explaining what they did exactly, they simply write that they used two statistical models to adjust for preterm birth and that the “second adjustment” showed that there was a) no relationship between preterm birth and NDDs and b) there remained a relationship between vaccination and NDDs.

That is quite a statement. The researchers’ claim that there is no association between preterm birth and NDDs flies in the face of long-established science linking the two.

Anytime research that contradicts long-understood and scientifically documented relationships should be read with a critical eye toward the methods used. But that is almost impossible here, as the only documentation the authors offer of their methods is to say they carried out a variety of regressions using a statistical software program called SAS.

What makes this worse is that the authors did not explicitly offer a hypothesis or any statistical criteria prior to beginning the study. That makes their data vulnerable to confirmation bias, as it allows them to potentially slice and dice data in as many ways as they want and plug those data into a program to see if their desired results that pops out. This study’s small sample size undoubtedly compounds this possibility, as it increases the likelihood for chance associations that may be flashy, but are ultimately not reproducible.