The Concussion Blog

Is it a bombshell or is it just a plain dud?  I say bombshell, but not in a good way for anyone involved with this “research”.

Last week I was inundated with emails regarding this “new” research about mouth guards and concussions.  There were roughly 16 emails in a one hour time span; some wanting comment, some telling me I have been wrong all along, some promoting the research.  This was a “huge” development in my area and my little corner in the blogosphere.  To fully understand perhaps some history is needed (“mouth gear” search on this blog) when it comes to my feelings on mouth gear and concussions.  Here are some selected comments attributed to me;

• For a health care professional dealing with sports injuries this should be a telling statement.  However there are those out there trying to push a product and make money on the misinformation about concussions.
• Rather, it is the insinuation of the marketing that makes these products seem as though they will prevent or even decrease concussive episodes.
• And to claim that there are devices out there that WILL reduce concussions is just as dangerous as not investigating the possibilities of it working.

The basic fundamentals we should be cognizant of here are: concussion is a BRAIN injury, the BRAIN floats inside skull, Physics dictates that the BRAIN will move depending on the forces applied to the skull/head (not always from a blow to that area), mouth gear cannot stop the BRAIN from moving, mouth gear cannot attenuate any forces to the skull/head that are not in the oral region, mouth gear does nothing for the skull/head when forces are placed on it in rotational, angular, acceleration or deceleration fashion.

Now that we have that all out-of-the-way this is the General Dentistry article I was asked to comment on.  On face value and from a “peer-reviewed” angle it seems all good.  A significant finding between custom mouth gear (noted as LM MG in article) and over the counter “boil and bite” mouth gear (noted as OTC MG).  However once you take a deeper look there are some peculiar problems, in my humble opinion – that comes later.

First, we should look at the possible limitations of this study that seems well populated and well thought out (honestly these were my first concerns before finding the real issue):

• Were the injuries controlled for by football position? (we have documented this issue here)
• Were the injuries controlled for by size of players/school they were playing?
• Were the injuries controlled for by playing time? (more exposure more risk)
• Were the injuries controlled for by game vs. practice?
• Were all the injuries seen and recorded by a single MD or was it the ATC at each school?
• Did any of the players have a previous history of concussion?
• Was the study controlled based on practice habits of the teams? (do some hit more than others)
• How do we know that every player complied with the “no wedging or chewing” rule? (this plays a massive role later)
• The study says that all 412 subjects wore the same exact helmet, I find that: A) hard to believe and B) was the fit on every player the relatively the same?
• Who funded this research? (no disclosure)

As you can see there is a litany of reasons I would have dismissed this research, if I were peer reviewing because those limitations are extremely real and realistic to control for in this type of study.  I wrote to the public relations group handling this research and was unable to get a straight answer on those questions I raised.  In the meantime I sent out the article to some better than average “stat nerds” and awaited a response.

While waiting I noticed something really troubling, as in a fatal flaw with the research.  In some places an oversight like this is intolerable, because letting this happen could set one up for producing “cooked” research.  I re-read the results section and saw this (emphasis mine):

One player was wearing a cut-off OTC MG with no support in the posterior occlusal area, and 2 players were not wearing an MG at the time of their injury; these 3 players were all in the OTC MG group.

The one player with the cut-off OTC MG disobeyed the instructions given by the lead author of this study, he modified the mouth guard.  This should have nullified the reporting of said injury.  The other two WERE NOT EVEN WEARING A MOUTH GUARD AT THE TIME OF INJURY!  Yet, they all were included in the results as an injury for the OTC MG group.

I may be some athletic trainer in the sticks of Illinois, but I do know a problem when I see one.  I feel that including all three of those injuries in the OTC MG concussion pool seems a bit off.  None of them complied with the study’s instructions, even though they were in a OTC MG “school”.  This study counts 16 concussions in that group, when in reality it was 13.  The “doubling” of concussions as touted by the research and the PR is both wrong and knowingly inaccurate.  If, and I mean IF, every other participant was following the protocol then the ratio was 13:8 not 2:1.

About that time I received the email from my “stat nerd” and he saw the same thing and gave me the following comments;

Only counting 14 concussions in the OTC MG group does one get a statistically significant result with chi-square adjusted single tail.  A very weak result.  All other measures the results are not significant.  With 13 concussions in the OTC MG group nothing is statistically significant.

Even if we were to include the cut-off mouth guard in the group only one stat run shows significance and it is not very strong to boot.  A more rigorous test like Fisher’s Exact Test two-tailed also shows no significance.  Below is a screen shot of the word document.

This is quite the information.  How do data get included in a supposed peer-reviewed article that is obviously not part of the group?  How do people who have many more degrees and letters behind their name let this type of information slip through?  But wait there is more…

When I first read the results I passed over, without thought, a note identifying that there were 23 concussions and a player received a second concussion but that was also included in the stats (bringing total to 24).  The article never identified which group, LM MG or OTC MG, was unlucky enough to have two concussions from one subject.  However, with what we know about concussions it should have been easily understood to probably not count that second one due to predisposing history.  If the second concussion was also in the OTC MG group that again raises some serious concerns and changes the statistics.

After all my questions the numbers just don’t add up; we could very well be looking at 12:8 (OTC:LM) rather than a 16:8 ratio.

(Quick aside here 24 concussions out of 412 participants (5.8%) is a very low number for high school football, perhaps someone should look into what they are doing in those schools to keep the numbers down!)

So for those of you wanting and waiting for comment on this article; well when you look at the real numbers it comments for itself.  The real “commenting” on this has to do with the “snake oil” type approach of cooked research.  It is hard to find out who would gain the most from this, as no funding disclosure was made available but bad science is bad science.

Remember mouth gear is great for protecting against oral and dental injuries and every participant in a collision sport should be wearing one – you should also be wearing one if you or your family has spent some good money on your teeth.

When it comes to concussions, mouth gear will not attenuate or prevent or make you less likely to get one, period.

====

12:26pm CST: Here’s a statement from Robert Cook, Executive Editor, General Dentistry:

General Dentistry is reviewing the study, “Role of mouthguards in reducing mild traumatic brain injury/concussion incidence in high school football athletes,” to evaluate the data presented. This study was not funded by industry and Dr. Winters has no disclosures.