Automated Primary Care Screening
for Mild Cognitive Impairment and Alzheimer’s Disease
Jane B. Tornatore, PhD[1], Emory Hill, PhD[1]
& Kenric W. Hammond, MD[2]
POSTER
Presented at The 8th International Conference on Alzheimer’s Disease
and Related Disorders, Stockholm, Sweden 2002.
ABSTRACT
Background: People with Mild Cognitive Impairment (MCI) appear to develop Alzheimer’s disease (AD) at a rate of 10-15% a year. Since most new treatments for dementia focus upon slowing the progression of AD, it is critical to test with a screen in primary care at an early stage fo the markers of future cognitive decline & the need for intensive diagnostic evaluation.
Objective: The Computer-Administered Neuropsychological Screen for Mild Cognitive Impairment (CANS-MCI), a self-administered touch screen battery, was designed to test (in primary care offices) for the earliest predictive signs of AD dementia. The CANS-MCI incorporates screening tests of all cognitive dimensions known to predict AD dementia: spatial relations, executive inhibitory functions, memory, & language fluency. The usability of the CANS-MCI tests in primary care offices (acceptability & relevance to subjects, absence of test anxiety & high rate of complete self-administration) was previously established.
Methods: 265 elderly volunteers are enrolled in a 3-year longitudinal NIA-funded study to test the CANS-MCI for screening test usability in primary care, validity & reliability. Findings from baseline MCI test screen data are presented.
Results: Internal consistency of the scales ranged from .515 to .966. One-month test-retest reliability correlation coefficients were all highly significant (p<.001). Concurrent validity correlation coefficients were also all highly significant (p<.001). A high level of diagnostic validity was attained using the criterion of performance on the WMS-R LMS II. Principal component factor analysis established 3 factors that explained 63% of the variance of scores: Recognition Retrieval/Language, Executive Functions & Episodic Memory Acquisition.
Conclusions: As effective treatments for AD emerge, it becomes important to screen people during primary care visits who have the earliest signs of the cognitive impairments most likely to become AD. The CANS-MCI is an easily administered, valuable screening tool in primary care to determine whether more intensive testing for early mild cognitive impairment and Alzheimer’s
disease is warranted.
INTRODUCTION
The concept of MCI both as distinct diagnostic entity & as a precursor to Alzheimer’s (AD) suggests that instruments focused upon MCI measurement would provide useful screening test information in primary care offices for decisions concerning full diagnostic evaluations for AD dementia.
No single cognitive/behavioral domain can be used to differentiate persons who will develop Alzheimer’s from those who will not. Several screening test dimensions, when combined, significantly enhance the predictive validity of a test battery because of variations in the initial cognitive deficits associated with early stages of AD. Furthermore, determination of the rate of change in at least two cognitive markers is a better means of predicting the development of AD dementia than is a single assessment. Repeated screening test measures are also necessary to detect changes in high functioning adults who have the cognitive reserve to compensate for early symptoms.
Current methods of Alzheimer’s detection in primary care are costly & often deferred until later in the disease process when interventions to delay MCI and AD are likely to be less effective. Therefore, an effective screening device for MCI would incorporate measures of multiple cognitive domains, measure changes over time, & be cost efficient.
METHODS
Subjects
A total of 265 elderly people were recruited through senior centers & retirement homes in 4 counties of Washington State. Exclusionary criteria were non-English speaking, significant hand tremor, inability to sustain a seated position for a minimum of 45 minutes, very recent surgery, cognitive side effects of drugs, indications of recent alcohol abuse, or inadequacies in visual acuity, hearing, or dominant hand agility.
Test Development
The CANS-MCI is a self-administered screening instrument that measures multiple cognitive domains & has the ability to measure changes over time. Development of the CANS-MCI tests was based upon findings of previous neuropsychological testing research.
The usability of the CANS-MCI by elderly subjects was previously established using the following criteria: acceptability, ease of administration, & completion of all tests entirely by self-administration. Even study volunteers with mild AD were able to complete the tests with minimal assistance. Moreover, the CANS-MCI appeared to offer a way of enhancing perceptions of control over testing & avoiding the activation of interpersonal defenses in primary care doctors’ offices.
Both the stimulus & response characteristics of the CANS-MCI are markedly different from traditional screening tests. The range of responses in the CANS-MCI is limited by the touch screen modality. However, in other populations these touch screen responses have been found to produce error rates similar to those produced with traditional verbal responses.
Instrument Description
The CANS-MCI presents the following progression of tasks to the subjects:
Introduction
- Presents progressively more difficult General Reaction Time tasks that prepare the subject for the first test.
.
Word/Picture Matching
- Presents 4 pictures of objects with one word, & the user is instructed to touch the picture that goes with the word.
.
Guided Recognition-Immediate
- User consecutively learns 5 sets of names of 4 pictures by touching pictures that fit into categories and then being told the name of the pictured object in that category.
- User is tested after each set of 4 pictures and the set is re-learned if mistakes were made.
- 20 3-button displays, each with an object name learned and 2 incorrect names from other categories are presented, with category-guided recall and re-acquisition of missed items.
.
Design Matching
- Presents 8 designs paired with letters in non-alphabetical order, & a set of 8 letters in alphabetical order.
- 1 of the designs appears in the middle of the screen, & the user is instructed to touch the letter paired with it.
- Complexity of attention-switching required is increased by within & between-test interference.
- Changes are made to the designs halfway through this test to present several types of interference.
.
Clock
- 10 clock blank faces are presented.
- A digital time is presented, & the user is instructed to first touch the hour hand position on the blank click face, then the minute hand position.
.
Stroop
- User quickly touches buttons matching names of colors presented.
- User is then instructed to touch buttons matching the ink color, not the word name, of the words “Red”, “Blue” or “Green”, presented one at a time in either red, blue, or green ink.
.
Picture Naming
- Pictures in multiple categories are presented, each with 4, 2-letter word beginnings, 1 naming the picture.
.
Guided Recognition-Delayed
- One additional recognition test trial, with guided recall for missed items.
Table 1. Internal Consistency (Alpha Coefficient Reliability)
| Test | # of Items | Coefficient Alpha |
|---|---|---|
| General Reaction Time | 10 | .810 |
| Design Matching (accuracy) | 136 | NA* |
| Clock Test (accuracy) | 30 | .896 |
| Stroop Discordant Item (reaction time) | 48 | .966 |
| Guided Recognition (accuracy) | ||
| Immediate † (5 Trials of 20 items) | 5 (trials) | .929 |
| Delayed (1Trial) | 20 | .616 |
| Immediate & Delayed (combined 6 Trials) | 6 | .928 |
| Guidance Efficacy | 5 | .568 |
| Picture Naming (accuracy) | 42 | .762 |
| Picture Naming (reaction time) | 42 | .793 |
| Word/Picture Matching (reaction time) | 14 | .871 |
Scores were only given for the items completed within the time limit. Participants did not all answer the same number of items so we were unable to perform reliability analyses.
† If any of the 20 correct items in a trial are not touched, the subject receives a guided recall test on that item.
Table 2. Test Re-Test Reliability
| Test | Time 1 Mean (SD) | Time 2 Mean (SD) | Coefficient Alpha |
|---|---|---|---|
| General Reaction Time | 0.77 (.21) | 0.73 (.17) | .702 |
| Design Matching (accuracy) | 38.05 (11.41) | 41.37 (8.94) | .765 |
| Clock Test (accuracy) | 30.65 (9.45) | 32.86 (8.89) | .792 |
| Stroop Discordant Item (reaction time) |
1.69 (.48) | 1.61 (.52) | .794 |
| Guided Recognition (accuracy) | |||
| Immediate | 17.73 (2.01) | 18.13 (1.86) | .681 |
| Delayed | 17.58 (2.27) | 17.83 (2.25) | .607 |
| Immediate & Delayed (combined) | 35.23 (4.21) | 35.95 (3.87) | .760 |
| Guidance Efficacy | 0.86 (.15) | 0.90 (.14) | .385 |
| Picture Naming (accuracy) | 31.59 (4.81) | 32.00 (4.91) | .788 |
| Picture Naming (reaction time) | 6.18 (2.13) | 5.90 (2.26) | .854 |
| Word/Picture Matching (reaction time) |
2.06 (.56) | 1.93 (.49) | .833 |
Table 3: Correlations of Standardized Tests with CANS-MCI Tests
| Conceptual Domain (CANS-MCI) | CANS-MCI Test | Standardized Test | Correlation Coefficient | P-value |
|---|---|---|---|---|
| Attention | General Reaction Time | Digit Symbol | -.585 | <.001 |
| Visuospatial ability | Design Matching (accuracy) | Digit Symbol | .537 | <.001 |
| Spatial relations | Clock (accuracy) | Digit Symbol | .469 | <.001 |
| Mental control | Stroop Discordant Item (latency) | Digit Symbol | -.565 | <.001 |
| Memory acquisition | Guided Recognition-Immediate | Mattis Memory WMS LMS-I |
.518 .540 |
<.001 <.001 |
| Guidance Efficacy | Mattis Memory WMS LMS-I |
.327 .352 |
<.001 <.001 |
|
| Memory retention | Guided Recognition-Delayed | Mattis Memory WMS LMS-II |
.447 .440 |
<.001 <.001 |
| Composite memory score | Guided Recognition-Immediate & Delayed | Mattis Memory WMS LMS-I WMS LMS-II |
.486 .519 .525 |
<.001 <.001 <.001 |
| Picture naming | Picture Naming (accuracy) | Mattis Initiation | .584 | <.001 |
| Picture Naming (latency) | Mattis Initiation | .616 | <.001 | |
| Other fluency tests | Word/Picture Matching (latency) | Mattis Initiation Digit Symbol |
-.496 -.636 |
<.001 <.001 |
Table 4: Diagnostic Validation using Delayed Memory Criterion
| Variable | WMS-II ≤ 10% Mean (SD) | WMS-II > 10% Mean (SD) | P-value |
|---|---|---|---|
| N | 44 | 215 | |
| Demographics | |||
| Age | 80 (8.4) | 76 (8.4) | .01 |
| Years of formal education | 13 (3.1) | 15 (2.7) | .02 |
| CANS-MCI Tests | |||
| General Reaction Time | .91 (.28) | .73 (.17) | .000 |
| Design Matching (accuracy) | 29 (13.0) | 40 (10.1) | .000 |
| Clock (accuracy) | 24 (8.8) | 32 (8.9) | .000 |
| Stroop Discordant Item (latency) | 1.94 (.51) | 1.64 (.45) | .000 |
| Guided Recognition (accuracy) | |||
| Immediate | 15 (2.7) | 18 (1.4) | .000 |
| Delayed | 15 (3.2) | 18 (1.8) | .000 |
| Immediate & Delayed (combined) | 30 (6.4) | 36 (2.8) | .000 |
| Picture Naming (accuracy) | 27 (5.1) | 32 (4.3) | .000 |
| Picture Naming (latency) | 8.4 (3.1) | 5.8 (1.6) | .000 |
| Word/Picture Matching (latency) | 2.57 (.69) | 1.95 (.46) | .000 |
Table 5. Exploratory Factor Analysis (N=132)
| CANS-MCI Tests | Language/Spatial Fluency | Executive Function/Mental Control | Episodic Memory |
|---|---|---|---|
| General Reaction Time | -.294 | -.741 | .046 |
| Design Matching | .497 | .535 | .275 |
| Clock (accuracy) | .620 | .297 | .142 |
| Stroop Discordant Item (reaction time) | -.169 | -.791 | -.126 |
| Free Recognition-Immediate (accuracy) | .562 | .119 | .660 |
| Free Recognition- Delayed (accuracy) | .682 | -.120 | .492 |
| Picture Naming (accuracy) | .780 | .308 | .184 |
| Picture Naming (reaction time) | -.825 | -.242 | -.231 |
| Word/Picture Matching (reaction time) | -.543 | -.568 | -.167 |
| Standardized Tests | |||
| WMS-R LMS-I | .192 | .293 | .819 |
| WMS-R LMS-II | .211 | .320 | .783 |
| Mattis Initiation | .648 | .233 | .361 |
| Mattis Memory | .295 | .095 | .734 |
| WAIS Digit Symbol | .377 | .635 | .301 |
STATISTICAL ANALYSES
Reliability: Internal consistency: Alpha coefficient reliabilities; Test-retest: Pearson correlations.
Validity: Concurrent: Pearson correlations with the scores on previously validated measures to provide a standard against which the component tests could be assessed.
Diagnostic: T-tests used to analyze differences between subjects in the lowest 10th percentile of cognitive functioning & those in the highest 90th percentile based on WMS-R LMS II scores.
Factor Analysis: Exploratory principal components factor analysis with Varimax rotation & Kaiser normalization.
Confirmatory factor analysis: presented at the American Association of Geriatric Psychiatry Convention, March, 2003.
RESULTS
Internal Consistency: Only 2 tests, Guided Recognition-Delayed (accuracy) & Guided Recognition-Guidance Efficacy, did not meet the predetermined standard for internal consistency (alpha >= .70) Other alpha coefficients ranged from .76-.97 (Table 1).
1 Month Test-Retest Reliability: Correlations over a 1-month period ranged from .607-.854 (Table 2). All but 3 had scores over .70. Separately, the Guided Recognition Immediate & Delayed accuracy tests had alphas below .70. When the immediate & delayed recall tests were combined to form a more global memory measure, the alpha was an acceptable .76. Guidance Efficacy had a very low test-retest alpha. Because Guidance Efficacy scored below the cut-off criteria in both inter-item & test-retest reliabilities, it was not included in further analyses.
Concurrent Validity: The correlations between the CANS-MCI & the previously standardized measures were moderate but all highly significant. Correlation coefficients ranged from .440 to .636 (p<.001) (Table 3).
Diagnostic Validation: Groups of impaired & intact memory subjects were established to assess the degree to which the CANS-MCI was able to detect impairments in cognitive abilities that are diagnostic of MCI or AD. Significant differences were observed between the memory intact group & the memory-impaired group on all CANS-MCI subtests (p<.001) (Table 4).
Factor Analysis: Results suggest a 3-factor solution that explained 63% of the total variance. The factors were Recognition Retrieval/Language, Executive Functions & Episodic Memory Acquisition (Table 5).
DISCUSSION
Reliability: The CANS-MCI demonstrates a high degree of internal consistency & test-retest reliability. These measures of test stability are comparable to those of the standardized comparison tests. Thus, the CANS-MCI can be reliably used at one or multiple testing sessions. Slight improvements in the mean scores are evident on all tests over the one-month period, probably
due to the reduction of anticipatory anxiety & establishment of positive relationships with the participants.
Validity: Cross validation of the CANS-MCI with the WMS-R LMSI & II, WAIS Digit Symbol & Mattis subscales demonstrates that the CANS-MCI subtests produce meaningful score differentiation of the memory impaired & non-memory impaired elderly. This is confirmed by an analysis based upon a WMS-R LMS II diagnostic criterion.
Factors: The factor analysis indicated that CANS-MCI items loaded onto 3 main factors: Recognition Retrieval/Language, Executive Functions, & Episodic Memory Acquisition. Design Matching & Word/Picture Matching loaded heaviest on Executive Functions but also heavily on Recognition Retrieval/Language, reflecting the overlap of cognitive domains when recognition ability is measured with psychomotor speed tests. Immediate & Delayed Recognition loaded most heavily on the Episodic Memory Acquisition factor but also loaded heavily on the Recognition Retrieval/Language factor.
CONCLUSION
As effective treatments for AD emerge, it will become important to identify people in primary care office visits who have the earliest signs of the cognitive impairments most likely to become AD. The CANS-MCI tests are reliable & differentiate memory impaired from normal elderly, as determined by the WMS-R LMS II. The CANS-MCI is an easily self-administered, valuable primary care screening tool for MCI to determine whether more intensive testing for cognitive impairment and possible dementia is warranted.
Footnotes
-
Screen, Inc. Seattle WA, USA
-
Department of Veterans Affairs, Seattle, WA, USA
-
Weschler Memory Scale-Revised Logical Memory Components I & II (WMS-R LMS I & II); Mattis Dementia Rating Subscales (Mattis)-Attention, Conceptualization, Inititation, Memory; Weschler Adult Intelligence Scale, Digit Symbol Component (WAIS)
