We have highlighted the work done by Stefan Duma of Virgina Tech on the STAR rating system for helmets. I have said that a proper system to help with informed buying is key for education and awareness purposes. I continue to believe that Duma’s work is a good start as it relates to this, however there are some flaws. The Concussion Blog was created to bring awareness on all fronts, whether I agree with it or not. There are always opposing view points and we are willing to listen to all of them, the comment section is a prime example of this.
Staying with the goals of the blog I received an op-ed piece from Schutt about the STAR rating system, I feel it is both important to publish and reflect on the information. Below is the full article along with contact information if you have questions;
Why the VA Tech STAR Rating System Can’t Be Trusted
By: Glenn Beckmann
Professor Stefan Duma and a team from Virginia Tech University recently developed what they claim to be a football helmet testing process that shows how well a helmet can prevent concussions. This new STAR rating system is presumptuous, possibly irresponsible and probably dangerous. Why? Because the testing methods are severely lacking and the laboratory results do not accurately reflect real, on-field statistics – which discredits the ratings from the start.
There are dozens of factors that can cause a concussion, but the STAR rating system considers only one: linear forces of impact. Such a limited testing protocol ignores many of the other factors that have been widely recognized to cause concussions.
Some of the unaddressed factors include rotational forces; time duration of impact; position played on the field; temperature; medical history; height, weight, size and speed of player; genetics, etc.
Of these factors, the two most important are rotational forces and time duration of impact. Rotational forces are recognized by concussion experts around the world – and acknowledged in Professor Duma’s findings – as a primary causation of concussions.
How can a study (that is supposed to be a true indicator of concussion prevention) ignore primary factors in the causation of concussions?
But, let’s assume for a moment that the study is a true indicator of concussion prevention. It would stand to reason that the players wearing the top-rated helmets from the STAR system would be suffering fewer concussions.
That seems like a simple, logical conclusion, yes?
Unfortunately for the STAR system, that’s not the case.
Our team at Schutt Sports has collected and studied data for the 2009 and 2010 NFL seasons. We know which helmet every player in the NFL is wearing. And using the NFL’s Official Injury Report each week, we know which players are suffering concussions. After reviewing both seasons, here’s what we found:
- Riddell Revolution Speed (5-star rated helmet): 30 concussions among 302 players wearing it
- Schutt AiR XP (3-star rated helmet): 28 concussions among 298 players wearing it
- Schutt AiR Advantage (2-star rated helmet): 32 concussions among 487 players wearing it
According to the STAR system, the players wearing the XP and AiR Advantage should be suffering concussions at a significantly higher rate. But that’s simply not true. This is only a small sample of real, on-field injuries not following the predictions/ratings of the STAR system.
Advocates of the study want to know: why can we use rating systems to help us pick our cars, our doctors and our schools – but not our helmets?
Here’s the answer: because such a rating system gives a false sense of security to the coaches, parents, players and managers expecting a certain level of protection and, unfortunately, concussion deterrence, from helmets that are highly rated by the STAR system.
Already, that false sense of security is in play. A recent article in the Indianapolis Star told the story of high school football coach Ryan Gallogly learning of the STAR rating system and then feeling a great sense of relief when he found all his players were wearing four- and five-star helmets.
It would stand to reason that Schutt would be excited about this report – as our helmets are among those that Gallogly is so proud to have his players wearing. But no helmet should be praised simply because of the STAR system. Concussions are too complex of an injury to be whittled down to one simplistic, limited test like this one.
It is not my intention to discourage the academic research conducted by talented doctors, scientists and researchers like Professor Duma and his team at Virginia Tech. I just want to make sure they are sending the right messages and are not trying to capitalize on a frenzied concussion environment, at the potential expense of football players everywhere, of all ages.
When considering which helmets to use, coaches, parents, players and managers should reference the guidelines issued on the website for the National Operating Committee for Standards on Athletic Equipment (NOCSAE): www.nocsae.org
###
Glenn Beckmann is the marketing communications manager for Schutt Sports. He will send the comprehensive NFL data cited in the op-ed to anyone interested in reviewing. You can reach him at gbeckmann@schutt-sports.com.
After this post Dr. Duma asked that I post this in response;
Dear Mr. Beckman,
Thank you for your comments. Over the past few months, I have had discussions with Schutt representatives. Unfortunately, it appears those comments were not shared with you. Let me elaborate on two key issues that your raise:First, let us discuss rotational acceleration. It is important to note that the primary reason that we did not use rotational acceleration was because we wanted to develop a system that the manufacturers could quickly implement in their laboratories. Ironically, you direct players and coaches to NOCSAE, but NOCSAE only uses linear acceleration as well. In fact, no standard currently for any helmet or automobile uses rotational acceleration. Of course this is a limitation, and given the feedback we have received, we are developing STAR-2. We will release this in 2012 and it will include both rotational and linear acceleration components, as well as other testing modifications.
Furthermore, our data show that linear and rotational head accelerations are highly correlated in the majority of helmeted head impacts, especially the higher impacts. In general, for most impacts, lowering linear acceleration will lower rotational acceleration.
Second, it is not possible to draw conclusions from the concussion numbers and helmet use data that you present from the NFL. One can only do this if you know the head impact exposure of all players, so that you compare apples to apples. In other words, you can only compare on-field concussion rates in players with different helmets if you know the exposure of each group of players (ie, how many head impacts and at what severity). For example, if Helmet #1 is worn by starting linebackers in the NFL and they have 5 total concussions in one year, while Helmet #2 is worn by backup quarterbacks who sustain 0 concussions that year, it is incorrect to say that Helmet #2 is better than Helmet #1 since the exposure for those groups is very different.
Another problem with using the NFL injury data is that you effectively have 32 teams with 32 different policies on reporting concussions. As an illustration of this, let us look at the 2010 NFL injury report and four teams that had nearly the same types of helmets: Tampa Bay, San Diego, Seattle, and Baltimore. During 2010, Tampa Bay reported 0 concussions, San Diego reported 1 concussion, while Seattle reported 8 and Baltimore reported 7. This of course is
not reasonable, even without considering under-reporting issues, and it is a critical flaw in your conclusions.Finally, we are very careful never to use the term ‘prevent’ relative to concussions. Our rating system clearly talks about reducing the risk, and we never say any helmet is concussion proof. We only talk about reducing risk, and that is a key issue. This all boils down to one key question: are some helmets better at attenuating the impact energy and thus lowering head acceleration compared to other helmets? Our answer is yes, some helmet are better than others.
All of this information and more can be found on our web page under frequently asked questions. http://www.sbes.vt.edu/nid.php
Sincerely,
Stefan Duma
The Concussion Blog 
And with that false sense of security, there is probably also a corresponding tendency to downplay a concussion when it happens.
These sensor studies drive me bats. Broglio and Guskiwicz having been doing these studies for years. If one inspects number of reported concussions vs. the readings on these sensors it implies either the readings are faulty or the diagnosis of concussion is faulty or a mixture of error intrudced in both.
There is no good simulation model for brain injury, mechnical or computerized. No model offers sound inferences regarding the translation of forces on the exterior of the skull to the damage done within the skull. Given the complex dynamic forces reigistered during a concussion, the protection offered by a particular helmets cannot be predicted by any method currently avaialble.
On the other, sensor studies are very popular with various funding sources.
So what is your point Joe? Because no good simulation exists *today* then we should not bother seeking a model that does work? Because sensor/reporting models have been flawed in the past, we should abandon any use of technology going forward?
Let’s not forget the words of our friend Mr. Edison, “I have not failed 700 times. I have not failed once. I have succeeded in proving that those 700 ways will not work. When I have eliminated the ways that will not work, I will find the way that will work.”
I don’t disagree but presenting these studies if they, in fact, were some leap forward instead of failed works-in-progress.
There is a real need to produce robust models and instead studies of little or no utility with PIs who have no qualifications in system dynamics or mechanical biology are funded so no progress is made.
Sensor studies as such are of no value.
Professor Duma,
Contrary to your assertion, the discussions you’ve had with representatives of Schutt have been made known to me. In fact, all of the people you’ve had those discussions with contributed to the writing of this op-ed piece. In fact, I was at the University of North Carolina when you presented these same arguments against our reservations at the commencement of the National Sports Concussion Cooperative.
I was there in North Carolina when NOCSAE reiterated their published warning of the STAR system, cautioning that there are still too many “ifs” to advise athletes and parents to rely only the rankings of the STAR system.
You state in your response above that ” Finally, we are very careful never to use the term ‘prevent’ relative to concussions. Our rating system clearly talks about reducing the risk….”
That may be a fine line of distinction in the world of laboratories, but in the world in which people are buying and playing in football helmets, the STAR System is just as clearly seen and being positioned as a tool with which parents, players and coaches can make conclusive decisions about which helmets to choose.
In fact, according to the methodology you published with the STAR system: “the STAR value is the number of concussions that one player may experience through the duration of playing one complete season with a specific helmet.”
In the real world, that is a difference without distinction. You talk about the reduction of risk of concussion while a typical lay person will see the above words and think “I’m going to suffer fewer concussions wearing a 5-star helmet than I would wearing a 3-star helmet.” How is that not an unspoken promise of concussion prevention?
At North Carolina, there were quite a few experts who reacted to your study thusly: it’s a good start and something is better than nothing. You did not deter them from such statements and, in fact, you echoed those sentiments in your own presentation, acknowledging your research was incomplete and had limitations but it was a good start.
Yet, when we broach the topic of real, on-field NFL injuries, you say “You can’t use that. The information is not complete and has limitations.” I’m sure you’re correct, although we will disagree on the magnitude. These are the official injury reports of the National Football League. There are, literally, millions of dollars riding each week on accurate information being distributed by the league, including injury information. If this data was so fundamentally flawed, as you imply, myriad external forces would pressure the NFL to correct it. And quickly.
But, for the sake of argument, let’s say the official NFL injury reports are flawed. Is it not a fair argument for us to say “That may be so, but it’s a good start and something is better than nothing?” After all, isn’t that one of the foundational components of your claim that the STAR system is a sufficiently accurate tool by which to judge helmets? You admit your study does not measure some primary concussion factors and yet you claim it to be a viable way to judge the concussion performance of helmets. Which of us is the Goose and which of us is the Gander?
As far as rotational forces are concerned, the breadth of research we have been exposed to indicates that rotational forces are almost certainly more important in concussive incidents than the linear forces that you are measuring. You may wind up being correct about the relationship between linear forces and rotational forces, but so far, most of the scientific and medical community holds a different position.
So we will stick with our assertion that any tool that purports to measure concussive incidents or the risk of concussive incidents and ignores primary factors such as rotational forces or time duration of impact is a very limited tool from which broad conclusions cannot reliably be drawn.
Above all, and the most un-scientific of all our reservations and one you did not address, is the false sense of security that the STAR system has generated and will continue to generate in coaches, players and parents across the country. They will falsely believe that their players are going to suffer fewer concussions because they have the “highly rated helmets.” Will they, as a consequence, let down their vigilance against concussions and head injuries because of that security? How are we to know?
We will continue to believe that your research is a valuable step forward in our battle against concussions and head injuries but that the STAR system is not an accurate enough tool from which to draw conclusions about helmet performance. You are trying to simplify a very complex injury into a single, easy-to-digest measurement. It is an admirable and worthwhile goal but we we believe your research might be moving faster at the expense of better.
Glenn Beckmann
Schutt Sports
Do others feel as if Beckmann plays second fiddle to Dr. Ferrara of the Xenith?
Ferrara and Xenith publicly point out flaws/misconceptions in the V-Tech Study and how to improve it, then Beckmann and Schutt follow.
Ferrara and Xenith point out the need for more regulation of reconditioning, then Beckmann and Schutt follow.
Ferrara and Xenith publish document demonstrating need for safe tackling technique, rule enforcement, education, Beckmann and Schutt follow with similar claims.
I feel like the Xenith always takes the first mover proactive approach, then Beckmann, Schutt respond.
Just my take. Cool blog.
Coach B.
Coach,
If that were true, why was Xenith not at the news conference announcing the National Sports Concussion Cooperative and Duma’s work?
I agree with you that Vin Ferrara and Xenith have many of the same positions on the VA Tech Study as we do. I will disagree that they have been first, so to speak, in opposition to the VA Tech study and been more prolific in their opposition. But I’m not sure that matters. I could just as easily tell you that they have followed us on many matters.
If that’s important to you, fine. But Schutt has always taken the position that helmets are not the answer to concussion problems and that helmets cannot demonstrably prove they significantly reduce or eliminate concussions. Since years before the day Vin squeezed the famous air bottle in his bathroom. And unlike Xenith, we have never made claims about our helmets’ ability to reduce concussions. And never will.
In terms of your claims about Xenith and reconditioning – you could not be more wrong. Xenith’s position on recon is filled with fear-mongering and hyperbole. If Dustin Fink – the moderator of this blog – would like that conversation to take place, we can do it on a separate thread.
Glenn Beckmann
Schutt Sports
I tend to agree with Coach Bart here and I find this blog interesting. I also made a comment about the Austin Collie section. I do not know who Mr. Beckman is but I feel that Xenith seemingly has been doing a lot with regards to this new “concussion movement”. I almost feel like there has been a momentum shift within the coach, parent and ATC communities. From my perspective it seems as if Xenith challenges some of the older ways of thinking and it somewhat forces hand for change. As I stated, only time will tell what comes of all of this, but its interesting seeing a smaller company be so relevant so quickly. Kind of David Vs Goliath-ish, Xenith/ Riddell.
Being in the tech industry it also reminds me a bit of Apple Computers in the early stages.
MS
As much scrutiny is needed in the area of oral protection. Cantu has made it clear, helmets have their limitations, both in research and in play. Standard boil and bite mouth guards also have limits and may cause more of a problem in a patient with prior temporal mandibular joint dysfunction. How this relates to helmets, all helmets affix to the chin, causing rotational torque on the tmj. Researching different methods of construction and materials is paramount. Protecting these forces from contacting the temporalis nerve is crucial. Other related areas of concern, the other seven cranial nerves within the temporal mandibular joint. Until this becomes part of the research model, including helmet chin strap design’s, the mystery of concussion will be just that.
To Glenn,
I understand that you are close competitors of Xenith, but I would not go as far as to insult an individual/ inventor of a product so highly touted by the medical community. Coming from a market competitor it comes off disrespectful and defensive. As I believe Dr. Ferrara received a M.D and degrees from prestigious schools, Columbia or Harvard, I believe he has more then earned this. Judging from the positive experiences that many players, coaches and parents have shared with me about Xenith, it is very apparent that they are building/creating market and thought leadership within the concussion space.
To Mark S.
I share a similar viewpoint with regards to the smaller company. In many cases the smaller more agile company can be more laser focused on the issues that need to be improved upon, i.e helmet technology.I look forward to more comprehensive testing protocals on current and future products.