Jacob Resch an assistant professor at the University of Texas – Arlington just presented his findings about ImPACT testing and its reliability. I was privy to a media release of the information, here it is in full (bold is my emphasis);
Media contact: Traci Peterson, (817) 272-9208, tpeterso@uta.edu
ARLINGTON – A computerized testing system commonly used to evaluate sports-related concussions misclassified up to 29 percent of healthy participants in a recent test by a University of Texas at Arlington kinesiology researcher.
The results from the study of the ImPACT computerized neuropsychological testing system emphasize the need for multiple types of assessments, said Jacob Resch, an assistant professor of kinesiology and director of the University’s Brain Injury Laboratory. During the study, the system had “only poor to good reliability” in 45 healthy participants, he said.
Resch’s findings were presented this week at the National Athletic Trainers’ Association annual meeting and clinical symposia in New Orleans.
Concussion assessment and management has become a national concern in recent years as former National Football League players and their families have come forward with personal tales of the lasting impact of hard hits. Legislators, national sports organizations and parents of children seriously injured have called for a cultural change in how brain injuries are diagnosed and treated.
Starting in August, public school athletes in Texas who sustain a concussion will not be allowed to return to play for at least 24 hours and must be cleared by a licensed healthcare professional. The University Interscholastic League strengthened the concussion management protocols after its medical advisory committee recommended the changes. Before, students with concussions could have been allowed to return to play if symptoms of a concussion cleared after 15 minutes.
The use of the ImPACT test, alongside other measures, is widely viewed as a useful tool in evaluating when a player may return to the field. Resch said the results of the study shouldn’t be interpreted as an indictment of the test.
“What our results stress is that when developing a sport-related concussion management protocol, a multi-facet approach including self-reported symptomology, a balance assessment and computerized neuropsychological testing should be implemented,” Resch said. “Ultimately, by incorporating this approach to concussion management we are significantly reducing the risk of returning an athlete back to the playing field too soon after a concussion which may lead to catastrophic consequences.”
The ImPACT test, which stands for Immediate Post-Concussion Assessment and Cognitive Testing, was developed in the early 1990s. It has five main outcome measures – composite and verbal memory, visual memory, visual motor speed, reaction time and impulse control. It is used for both baseline and post-concussion testing.
Resch, who is now using the ImPACT test in a lab study of North Texas high school athletes at risk for concussion, said researchers wanted to add to the limited scientific examination of the ImPACT test’s reliability. This is especially important as state and local entities consider making baseline testing required practice.
The healthy participants in Resch’s study had an average age of about 22. They were tested once, then again about 45 days later and again about 50 days later. The time periods are similar to what would be used in a clinical setting. The researchers found that ImPACT misclassified22.2 percent to 28.9 percent of the healthy study participants on one or more test variables at the 45 and 50 day marks. Those who were misclassified did either significantly better or significantly worse than their baseline, even though their health had not changed.
Resch began to study the test’s reliability while at The University of Georgia. Co-authors on an abstract presented at the conference include University of Georgia faculty and researchers from the Shepherd Center in Atlanta and Georgia Neurological Surgery in Athens, Ga.
“Research like this reinforces the need to look critically at what we do and make it better,” said Dr. Kimberly Walpert, coauthor of the study and a partner at Georgia Neurological Surgery. “I think this is particularly important in view of the growing evidence regarding how multiple, seemingly insignificant brain injuries may have long term consequences for athletes.”
Resch hopes presenting the study data at the national conference will encourage ImPACT’s parent company to improve test-retest reliability and motivate athletic officials to continue using additional methods – like balance assessment and interviews with the athlete.
“We are very excited about the work that Dr. Resch is doing,” said Professor Lou Fincher, chairwoman of UT Arlington’s kinesiology department. “This study, as well as his current and pending projects, will continue to help shape the standard of care for assessing and managing concussions.”
The University of Texas at Arlington is a comprehensive research institution of 33,800 students in the heart of NorthTexas. Visit www.uta.edu to learn more.
All of this information is what we have been saying here for a very long time, and it is the crux of the tone Matt Chaney writes with in regards to concussion reform. We cannot, as clinicians/parents/athletes, expect any tool to be specific and sensitive enough for every single person. The human brain is so complex that creating a “gold standard” for evaluation would be nearly impossible, in my opinion.
Remember that if you have the resources every tool that is available will help with the concussion issue. Not unlike working on a car or a remodel of a house, the better and more options you have to get the job done the better the result. As always, I will remind you that I do use NP testing, in fact I find it valuable but not for the “clearance” aspect of it. I like to use these products as an awareness device, as well as a confirmation that the injured individual is NOT READY. Personally as an athletic trainer I don’t think I have ever cited, nor will I in the future, the passing of the NP test as an indication of readiness for play. Rather I rely upon the difficult task of a clinical evaluation, balance assessment, and reaction to physical activity for the best recommendations for the MD.
The Concussion Blog 
The deficincies of ImPact have been long known. Lest we not forget Joe Maroon, Mark Lovell and Mickey Collins, the so-called brains, behind ImPact denied the association with multiple head traumas with long term neurological injuries. ImPact has relied on its cosy ties with the NFL and other professional leagues that have a very defined agenda to dupe gullible uniformed ATCs and parents into using a substandard instrument.
The only method of fixing ImPact is replacing it with instrument built around sound science not the avarice of its devleopers. Computer Neuropysch testing is not the problem, it is Messrs Collins, Lovell and Maroon.
Interesting results. The work of Phil shatz in phila tells a different story with solid reliability. Sounds like you have something against the doctors who made the test. Despite this I think we all can agree that using just one test whether it be balance cognitive or self report is not ideal. Don’t throw the baby out with the bath water especially based on ad hominem reasons.
This is not an ad hominem assault. Messrs. Lovell and Maroon served on the disgraced NFL Concussion Committee. Please read some of those cooked research results. Lovell and Maroon sold an intsrument they owned to the NFL serving on its concussion committee and selected the people to interpret the results (see Peter Keating’s articles on ESPN). Why doesn’t “concussed” review the deaths of Mike Webster, Long, Strylcyk et al. all under Lovell and Maroon’s care in Pittsburgh. The bogus Riddell study conducted to promote dubious helmets supervised by Maroon and Lovell. Magic concussion preventing helmets, please!!!
How does Maroon justify selling fountain of youth drugs to the aging, that is, PEDs and still treat professional athletes?
Please reference facts, the Marine’s quickly dumped ImPact once its limitations became clear in the absence of self interested ImPact promoters.
Joe and Mark aren’t stupid just greedy and unconcerned with the health of athletes. Look at their 1998 book and they knew that multiple concussions caused long-term neurological damage. On the other hand, they knew they could not sell ImPact if he told the truth about it, validation was paid by ImPact or one of its partners, or head injuries. So rather than offend their masters at the NFL, NCAA and WWE, they simply used them to make money by duping high school and collge coaches/ATCs while failing to be medical professionals. Just make sure high value atheltes like Rogers. Rothlisberger et al pass ImPact in impossible windows of recovery so TV ratings are maintained and prep athletes will risk their necks to emulate their heros.
Here are the facts. You are correct that Maroon was the Steeler’s neurosurgeon in the 70’s, and he was charged with assessing athletes who were concussed. However, many of those athletes reported that they were fine when in fact they were not. There was no objective test at that time to assess concussion and return to play. RTP was based on symptom report. This is why he brought in Lovell to create an objective instrument instead of self report. ImPACT only became available in the late 90’s. So in fact if they had ImPACT back then those tragic cases might have prevented, if you believe in CTE. Those athletes were the reason that an objective test was created. Of course they were on the NFL committee because they were the only ones who were trying to address the problem.
In regards to so called “cooked” research; which studies are you referring to? There have been over 200 published peer reviewed articles about ImPACT, and the majority of them show good reliability and validity; many of which were conducted independently.
As for Keating’s articles, he is only telling half the story and takes only the information that he needs for his opinion. He is in the business of creating, not reporting, stories and this is well known.
As for the Marines, I dont know if you should judge how well something works by if the government uses it. However, the truth of the matter is that there are several branches of the armed services who use ImPACT. The Marines dumped ImPACT not because of it’s inadequacy, but because they have their own instrument they want to use, ANAM. This is a test they created and are desperately trying to make it work.
In regards to the Riddel study. That study did show positive results, however, had significant limitations which they discussed. They never said it was concussion proof, and they dont promote it. If you talk with Lovell, he will tell you there is no concussion proof helmet.
Please do not tell me to reference the facts when it is you who have not set forth any fact. For example, one of the quarterbacks you listed was pulled from a game by Dr. Lovell because he did not pass his test. Which appears to fly in the face of what you are purporting. You have not presenting one piece of factual information backing your point. Do a literature review of ImPACT and see that there is significant data behind it by non-partisian research, and not just some guys evil scheme to make money. It is not a dichotomous test but it is helpful in managing this serious injury.
Keep it coming fellas… This is a GREAT discussion…
Do you work for ImPact? If you believe in CTE or knew anything about it, you would know the likely cause is sub-concussive injuries not episodic concussion. ImPact has saved no one from CTE as sub-concussive events are not measurable with ImPact, in fact, concussive events are questionably measured by ImPact. Joe still denies that players who have died had a concussion.
Many people have and are dealing with the issue that have no affiliation with ImPact including Cantu, Barth, Erlanger, and Jordan to name a few. On the other hand, these scientists are not snake oil salesmen.
As far as cooked research, please refer to any studies published by the NFL concussion committee from 1995 to 2009 under Ira Casson, Viano, Maroon and the rest of the gang. To deny the long-term neurological problems associated with m-TBI either displayed ignorance or mercenary intent. While Lovell is not the brightest bulb on the string, Maroon knows and knew better. As far as independence, who paid for the studies and what was the affiliation to ImPact. It gets a bit murky. Did you get a copy of their 1998 book and compare it to statements made to the press and promotional materials of ImPact; bet you didn’t.
As ESPN is closely aligned with the NFL, it seems a bit of a stretch to suggest Keating engaged in creative writing. Perhaps, one does not know the meaning of conflict of interest. Dr. Lovell and Maroon selected their own instrument and then selected the doctors to interpret it while they were on the NFL concussion committee. Maroon and Lovell then promoted the NFL’s use of ImPact to sell it to unsuspecting high schools and colleges. That is a fact. Even the current NFL committee as sad as it is has removed the mandate to use ImPact. Thankfully, they removed Lovell as well. Dr. Barr’s commentary in the Casson article should scare every parent and trainer.
The military has made its view clear. It is spinning down the use of ANAM as it is one of the few tests that is even more laughably bad then ImPact. Nonetheless, don’t expect to see ImPact widely deployed as it is not capable of measuring blast injuries.
The facts of Riddell study are subject to any Federal Trade Commission investigation and Sen. Tom Udall of NM is looking closely into the matter because the study “which was cooked” implies a helmet substantially reduced the incidence of concussion in high school athletes. The study was conducted without proper controls with ImPact and use Riddell helmets. Guess who conducted the studies, Lovell, Maroon and the head of Research for Riddell. The video promoting the results was viewed millions of times. Now Joe says, we need more research. I am sure conducted by UPMC to promote the further use of ImPact.
I hope you are not relying on ImPact to determine whether a loved one should return-to-play. Then again, 20 years from now when that kid how got hit and put back and develops neurological cin the game passed ImPAct
Actually, there still is a big push by the military for ANAM. I work at a rather large military hospital, and there are orders to continue with ANAM. I recently came back from Iraq, where I was tasked with “continued validation” of ANAM. We do use ImPACT as well. There are several projects where we are using ImPACT. It is definitely a step above ANAM. You speak of blast injuries, in theory the biomechanics of the injury are the same sans a direct blow; therfore, the effects would be measured the same. So I do not know how you could say that a neurocognitive test would not measure blast injuries. That does not make sense.
Just thought I would weigh in from a military perspective. Good discussion none the less.
Army Doc.
ANAM is a disaster please note: Automated Neuropsychological Assessment Metrics:
Repeated Assessment with Two Military Samples, Enota (US Navy). In 30 years of development, it still has not been shown to be reliable.
Any push you are seeing is based on politics and not based on science.
The problem with any sports concussion instrument is that one will generate neumerous false positives as sipmly being put in combat is sufficient to cause what would appear to be statiscally significant change in tests based on reaction time. Further, blast injuries are quite distinct from sports concussion, please refer to either Spect imaging or DTI studies which illustrate a much more diffuse pattern of injury. Sports instruments are a waste of time. As ASG Lt. Gen Schoomaker stated in congresional testimony, ANAM in no better than a coin flip and I am sure ImPact is equally useless in measuring blast injury. In fact, the Navy SG and AF SG concur with Shcoomaker.
One requires a much broader series of measures to capture the damage done in combat and the patients recovery.
No I do not work for ImPACT, but I do use it and I will defend it when people people who are misinformed or have an agenda putting for misinformation to others. In all your posts you do not cite any peer reviewed literature regarding the efficacy of ImPACT. It is the most well validated and reliable tool that we currently have in sports related concussion. It is surpassed other measures like ANAM, Cogsport, Headminders, etc.. It’s reliability has been demonstrated by Phill Schatz out of Temple in 1 year and 2 year increments. It’s validity demonstrated in numerous studies including several out of Dartmouth… What research do you have that finds it is not?.. I am not saying this test is perfect. But it is the best tool we have at present to look at performance on cognitive functioning which is affected post concussion. It is not a dichtomous tool, aka RTP or NOT, but if it was my son. I would want him taking ImPACT along with a full assessment.. Since you do not think ImPACT is viable, what would you suggest?
As for Armydoc, I think he proves your point, it is political. You made the statements the Marines dumped ImPACT. I wonder why, because it does not work or because of political reasons. FYI blast injuries and SPORTS CONCUSSIONS are diffuse injuries. You should go back and look at the research on this. There are no focal injuries in these two subset’s, that would be more indicative of a moderate or severe injury. Hence why ImPACT has merit, because it looks at Diffuse functioning…. Not sure what you are talking about here?
CTE, yes if you believe in it. There are NO, repeat NO, research studies that have found a causal relationship between multiple head injuries in football and CTE. There are theories and sure it is possible. It is very intuitive that if you hit your head enough times it could cause LT difficutlies. But there are huge confounds in all the studies, and HUGE selection bias. I refer you to the article by Paul McCrory, that discusses Cantu’s work. There is a lot of research to be done in this area. I agree there has to be some long term effects of multiple repeated trauma; however, we just do not know what that is at this point. And speaking of conflicts on interest.
You continually bring up this one study with Riddell helmets, neither lovell or maroon have ties with the company. Both do not promote any 1 helmet. However you should investigation the relationship with Bob Cantu, the one pushing CTE and Shutte.
The NFL committee is made up of individuals who all have ties to some form of concussion assessment or have some sort of interest. That is the nature of that committee. You put forth one side of this story, but here is what Dr. Casson said “My position is that there is not enough valid, reliable or objective scientific evidence at present to determine whether or not repeat head impacts in professional football result in long-term brain damage,” Another doctor stated that there are many physcians who agree with Casson; however, are reluctant to speak up due to misinterpretation and the repercussion of speaking out.
How do you suggest we evaluate this injury and manage it?
Not to put splinters in my butt, but I see both sides very clearly. NP testing is very well reliable and specific for about everything but concussions. There have been many articles calling into question the validity and reliability on ImPACT. The ones I can think off the top of my head are (not all authored together); Broglio, Randolph, Moser, Schatz, Echemendia, Solomon, Ott, Karpf…
But I also as an athletic trainer find some value of the NP tests for concussions, it is just a tool, and to be honest you probably need a neuropsych to interpret the results not AT’s, MD’s credentialed or not (just changed my stance on that after Echemendia spoke yesterday).
The issue with ImPACT specifically I believes comes from the marketing/press it has garnered. And once upon a time it was viewed as a clearance test, and still can come across that way in the media. What is clear to me with ImPACT or and NP test for concussions, is that if you rely on that and get thrown to the courts you are going to be in deep doo-doo. It is also a “money grab” situation as well with ImPACT, do you really need the baselines? Are they as important as they make it out to be? Yes it is wonderful to have a normative set of data for each individual, but what if you cannot afford it?
All questions that will be asked by some school some where…
Concussed:
I do believe in NP screening, if fact, both computerized NP screening and face-to-face paper pencil screening. ImPact simply not a very good instrument although better than ANAM. Cogsport/Axon and Headminder are both superior tests for sports concussion – check almost any publication. Both Cogsport and Headminder were created by serious scientists.
First, if a school can afford it get a trainer. Second, use SCAT and a very conservative protocol to return-to-paly. You might be sidelining a player for a month to three months. Next, if money and expertise is available use a quality NP instrument in the correct context. Proper baselining should be conducted annually, symptom windows observed, and rtp guidelines scrupulously followed. Players should be referred to a medical professional that understands concussions to make clearance decisions. Unfortunately too few MDs have the background to make the a sound judgment.
As far as blast injuries, it is simply beyond this forum to explain the differences but you should refer to Col. Geoff Ling’s of Darpa statements regarding the differences in injury. He is both a serious MD and researcher. I can assure you that these injuries are not remotely similar.
As far as CTE, left to be proven in Football, but it probably will be proven. Are you going to call into question the small arm results from Randolph’s study? Not a great study but it is something we have known for fifteen years. Football is positively associated with dementia and early impairment.
I find your attempt to sully Robert Cantu laughable. His thrity years of work are at an unusually high quality. His ethics have never been questioned in the scientific world. You may not agree with him but his contributions and insight are beyond question. He is an honest doc.
As far as the Riddell Study, Riddell paid for it, the instrument used was ImPact, the authors were Maroon, Lovell and the head of Riddell research. It was promoted so widely the Federal Trade Commission is conducting an investigation with good reason. I hope you go and look a all of Maroon, Lovell and Collins publications regarding ImPact. Check the conflicts that are not reported as required in the journals.
Casson is a Mexican medical school graduate in rhumetology not neurology or neurosurgery. His work is a joke. Since much of the funding for sports concussion came from the NFL or NCAA why do you think some many studies are so poor and who is responsible. Passing off conflicts as something we should expect is crazy. There are excellent researchers without conflicts that should have been tapped but the powers that be would find it difficult to control them.
I find it astonishing that anyone could defend anyone associated with the disgraced NFL concussion committee. I think there is still room for you at the flat earth society; I think they are still accepting members.
As more data becomes available expect lawsuits to fly. No more talk, no more uninformed opinions. Lawyers will hold sway and we will all be sorry for that. On the other hand, I will not be compelled to pay for the injuries.