You maybe heard about the recent retraction notice of a study published in Scientific Reports that raised concern about the safety of Gardasil(R) (HPV vaccine) and got retracted this week. Another anti-vaccine paper that bit the dust. I would have say, I am not surprised at all. Anti-vaccine studies have this very annoying habit of either being a proven fraud (remember the latest Shaw paper?), botching the experimental protocol with omission of proper controls (thats the Exley paper I have reviewed) or conveniently sweeping under the rug some data that are not fitting the narrative (that goes for one recent paper published by Gherardi).
But this one is interesting at several levels, because there is a bit of blood-brain barrier in it, and also adds to the list of papers retraction in Scientific Reports and recent threats by scientists in the editorial board to resign from their positions due to ambiguous and unjustified decision on a flawed paper (Disclaimer: I have a study authored in this journal and I have peer-reviewed for them a couple of times).
To be honest, I only heard about this paper during the last weekend and took some times to read the paper. I will be honest, I don’t see any scientific fraud in the sense of data manipulation. What concerns me is how such a botched study could even pass through peer-review process? Considering I self-impose a quality of standard in my manuscripts and still get challenged by peer-reviewers, seeing such junk studies getting a free pass is a bit vexing. I agree that open-access may not have the same stringency in terms of peer-review filter but considering Scientific Reports as part of Nature Publishing Group, you expect a rigor found for any Nature-related journals applied in this journal too.
But lets go through the paper, it is retracted but you can still access it here.
What is the wrong with this paper?
1. The experimental design in terms of groups
The first problem arises from Figure 1.
We have six groups: vehicle (PBS), pertussis toxin (PTX), Gardasil(R) (MSD, HPV vaccine 4-strains) (G), G+PTX, EAE and EAE+PTX. Let’s breakdown first these oddities. Why the author has included a PTX group, even more adding PTX with Gardasil? There is not much explanation in the text explaining the rationale (also the writing style is odd, very odd. I completely understand that the first author is not a Native English speaker. As an ESL myself, I completely understand that). Also, I am not aware about a higher risk on contracting pertussis upon vaccines.
The second aspect is the use of the EAE mouse model. EAE stands for experimental autoimmune encephalitis. It is the “gold standard” for a mouse model of multiple sclerosis (MS). The idea behind is to inject a brain protein (myelin basic protein or MBP), which will trigger an immune reaction (as the brain is a immune-privileged organ) and result in symptoms similar to MS. I would understand to compare the incidence of Gardasil(R) on MS patients by comparing EAE mice versus non-EAE mice but that is never the case (they even administer PTX to a sub-group).
So here we already start with a wrong experimental design: it just make no sense. A more rationale approach would have been the following:
Vehicle (PBS), Gardasil (G), EAE, EAE+G. That would have saved two groups and precious lives sacrificed in another useless study.
2. The experimental design in terms of statistics and power of analysis
Another important issue is the blatant dismissal of consideration of biostatistics and the power of analysis in the experimental design. For those that are not familiar with scientific research, you have to ensure you have a statistical meaning to your data to ensure the effect observed is real and not due to simple coincidence. This statement is especially true when working with vertebrates animals. Any animal experiment has to be approved by the institutional IACUC that ensure you have a clear idea of what are the purpose of your experiments, how you will ensure a humane treatment to animals and also have an optimal number of animals to achieve a statistical significance.
An important aspect for in vivo (animal) studies is to achieve at least a sample size of 8 or more animals per group (n=8). From Figure 1a, we have already a violation of this as only the G and the G+PTX have enough animals (n=14 and 21 respectively). All other groups are below the n=8 threshold. In addition, you have very different number of animals per groups (control has n=6, EAE has n=5) making the statistical power weak and also restricting the use of common statistical methods such as the use of ANOVA (ANOVA recommends that all groups have an equal number of samples).
3. The experimental procedure and treatment
This is where the firestorm came in: the experimental data. So lets bring this to the table: “Groups of 11 week-old female C57BL/6 mice were intramuscularly administrated 100 μ l of Gardasil or phosphate-buffered saline (PBS) for a total of five times. Ptx was intraperitoneally administrated 2 and 24 hours after immunization. The Gardasil vaccine or Ptx were administrated at 2- weeks or 4-week intervals“.
A key element in a paper is the methods section and this one utterly failed. Based on this information I have absolutely no idea why they injected five times (Gardasil immunization is maximum 3 times), why they injected the PTX right after the immunization (2 and 24 hours, suggesting a double-induction) and when they injected the Gardasil and PTX (how do they separate the 2-weeks versus the 4-weeks? When did they started?).
Also the use of 11 week-old female is not reflective of a human case scenario. If we approximate 1 human year to about 3.6 mice-days, you would expect to use young mice (males and females, to have a gender-balanced study) that are about 6-weeks old (~43.2 days). That would be about 12 years, the age of puberty.
Here we are basically injecting HPV to adult females, which is known to not provide an additional benefit as such population may have already been exposed to HPVs.
The next problem is the injection dose. It is 100microL, thats the equivalent of 0.1mL. A single dose of HPV is 0.5mL. Lets ignore the scale law and assume a mouse is an equivalent to a human. A 50th percentile weight at age 12 is about 40kgs. Lets assume a 6-weeks old mice to be about 20g.
If we assume 0.1mL injection to a 20g mouse, then the human dose-equivalent would be about 200 dose-equivalent injected at once! This is a serious issue because there is absolutely no chance that such things to happen in a lifetime (at grand maximum you may have 3-4 HPV injections, spaced in time). Also, if we assume the age scale, we are expecting to have mice receive their two doses within 48 to 96 hours, not 2 to 4-weeks.
I am not even entering the rationale to inject PTX right after immunization, which is utterly no-sense and just scramble defining the effect of HPV versus the effect of PTX. Another disastrous example of how this paper was flawed from the beginning.
4. Failure to report weight and clinical score
If we want to follow an EAE protocol, it is important to show the evolution of animal weight over a period of times (up to 15-21 days) as well as a clinical score. The clinical score is a well-described protocol in which features found in EAE mice are score from mild (tail flaccid) to severe (inability to move hindlimb or complete immobility).
These two graphs are almost present in any EAE paper outside in the literature.
I assume this is what Figure 1b and 1c wanted to show but very poorly. Indeed Figure 1c does not really show up anything. We dont know when these data have been taken, we have no idea about the onset time of symptoms and furthermore we have no indication of a statistical differences. This is already a waste of data.
5. The constant cherry-picking of the data and incomplete picture
The methods used are honestly laughable: some hematoxylin-eosin staining (a common histological stain that does not tell much unless you have massive brain damage or the growth brain tumor), Kluver-Barrera staining (for myelin staining), TUNEL stringing (for apoptosis), a behavioral test relegated in Supplementary Figure S1 (in which the author thinks that a P-value of 0.1 has a statistical meaning).
Where is the Evans blue extravasation staining to show a disrupted BBB? Where are the GFAP staining to show astrocytes activation? Where are the CD11b and F4/80 staining to show microglial cells activation and macrophages infiltration? Nowhere to be seen. We have to comptent ourselves with some miserable histological staining in Figure 2 and 3. Also no-one of the data about EAE is never shown past Figure 1. Figure 4 is even more laughable as the author only shows the staining of the G+PTX, giving the middle finger to the reader to how such staining looks like in vehicle, or G.
How can the author be confident that it was the Gardasil treatment, not the PTX treatment (despite being mentioned as a BBB disrupting toxin) being the sole contributor of all this?
Another anti-vaccine study, another case of botched science resulting in a junk paper, the sacrifice of animals over a useless experiment. That should not have been passing through the peer-review filter at all because of its deficiencies, yet was able to go through. If I was the reviewer behind it, I would have been ashamed to have this paper not outright rejected for major flaws in the study. Should we assume that the author recommended some complacent reviewers to this paper? Or should we question the integrity of the editorial board in accepting papers for publication that fail to address some scientific integrity? Again, anti-vaccine studies shows that they cannot challenge vaccine safety and can only make fool of themselves by producing junk studies like this time.
The first and senior authors of this paper produced a paper that is so bad, they should feel ashamed to even had published at first.