Concerning Cognitive Testing over the Internet

Concerning Cognitive Testing over the Internet

Cognitive testing over the Internet must be performed in controlled environments

Computer-assisted testing is better

Until fairly recently, all cognitive ability testing was done by professionals in their offices. It usually went without saying that the person being tested was not distracted, interrupted, or coached by anyone during testing. Recently, the internet has created the possibility of testing people wherever they are. There are a variety of computer-assisted test procedures that claim to be valid substitutes for direct, in-person professional testing.

The results of testing can be distorted by a person’s skill with the test interaction method. Certainly, the person being tested must speak the language of the test administrator. The person being tested must also be familiar with the method of responding that is required. Until recently, one could assume with paper-and-pencil tests that all testees were adequately familiar with all response methods – speech, touching the correct item with a finger, and the use of a pencil. However, a computer-administered test might require the use of a keyboard device, or even more problematically, a mouse. Individual differences in familiarity and proficiency with such response methods can drastically influence what are supposed to be differences in cognitive ability. Only speech and finger touches are adequately uniform and natural response methods when testing elderly people. Fortunately, computerized testing using touch screens and voice interactions can avoid the aforementioned problems.

Internet-based testing is not better

Not all computer-assisted testing is equivalent. Internet-based tests are now technologically feasible due to advancements in the speed, power and usability of the web, but although the Internet may appear to give the advantage of availability and ease of access — especially to physically disabled people — the reliability of Internet test results is markedly affected by two detrimental human factors: distractions and coaching.

The person setting up the test over the Internet is often the child of the person being tested, and less frequently the spouse. Typically this means testing is being done at home. There may be grandchildren running around. The television may be on in the background. The person being tested may be distracted by any number of things, or may simply leave the test to use the bathroom. Of particular concern is that the child or spouse often cannot resist helping in ways that distort the reliability of the results. For all these reasons, tests performed outside of a controlled environment, or without the assistance and supervision of a neutral test administrator, have to be considered invalid.

Of secondary concern is the speed and reliability of the Internet in the home. This is increasingly becoming less of a concern but without capturing diagnostic and performance metrics along with the test there is no certainty that the results will be valid.

Testing in a controlled environment is necessary

Computer-administered tests should be given by a neutral tester in a controlled environment, preferably a designated quiet room in a medical facility. Even then, the presence of a spouse or child is discouraged. The following is a demonstrative example. In the cognitive report shown below four testing sessions had been retrieved over an 8-month period.

When a patient is impaired to the degree that this person was on his first test session, Screen, Inc. recommends re-testing every 3 months. In the third session (August 27, 2014) this patient’s results were so distinctly different from the first two sessions, that the Screen, Inc. report writer called the testing facility (the doctor’s office) and asked if there had been anything different about the testing environment for that session. The support staff member said, “No. Well… his wife was in the testing room with him this time”. A re-testing was requested, with instructions to make sure that no test-taking assistance was possible. The results in the fourth session returned to the expected range.

Emory Hill, Ph.D.
Founder of Screen Inc., Dr. Hill has a PhD in Clinical Psychology, State University of New York at Buffalo. Later he completed an Informatics Fellowship (post-PhD) at the VA where he studied interface design, multimedia programming, user resistance, evaluation of adaptations to new medical record systems, and the implementation of automated medical records. A trained psychologist and psychometric specialist, Emory was in private practice for nearly 20 years. Before that, he served as an Assistant Professor of Psychology at SUNY, Brockport, NY.