Stockholm Excerpts

Stockholm Excerpts

Automated Primary Care Screening
for Mild Cognitive Impairment and Alzheimer’s Disease

Jane B. Tornatore, PhD[1], Emory Hill, PhD[1]

& Kenric W. Hammond, MD[2]

POSTER
Presented at The 8th International Conference on Alzheimer’s Disease
and Related Disorders, Stockholm, Sweden 2002.

ABSTRACT

Background: People with Mild Cognitive Impairment (MCI) appear to
develop Alzheimer’s disease (AD) at a rate of 10-15% a year. Since most new
treatments for dementia focus upon slowing the progression of AD, it is critical
to test with a screen in primary care at an early stage fo the markers of
future cognitive decline & the need for intensive diagnostic evaluation.

Objective: The Computer-Administered Neuropsychological Screen for
Mild Cognitive Impairment (CANS-MCI), a self-administered touch screen battery,
was designed to test (in primary care offices) for the earliest predictive
signs of AD dementia. The CANS-MCI incorporates screening tests of all cognitive
dimensions known to predict AD dementia: spatial relations, executive inhibitory
functions, memory, & language fluency. The usability of the CANS-MCI
tests in primary care offices (acceptability & relevance to subjects,
absence of test anxiety & high rate of complete self-administration) was
previously established.

Methods: 265 elderly volunteers are enrolled in a 3-year longitudinal
NIA-funded study to test the CANS-MCI for screening test usability in primary
care, validity & reliability. Findings from baseline MCI test screen
data are presented.

Results: Internal consistency of the scales ranged from .515 to .966.
One-month test-retest reliability correlation coefficients were all highly
significant (p<.001). Concurrent validity correlation coefficients were
also all highly significant (p<.001). A high level of diagnostic validity
was attained using the criterion of performance on the WMS-R LMS II. Principal
component factor analysis established 3 factors that explained 63% of the
variance of scores: Recognition Retrieval/Language, Executive Functions &
Episodic Memory Acquisition.

Conclusions: As effective treatments for AD emerge, it becomes important
to screen people during primary care visits who have the earliest signs of
the cognitive impairments most likely to become AD. The CANS-MCI is an easily
administered, valuable screening tool in primary care to determine whether
more intensive testing for early mild cognitive impairment and Alzheimer’s
disease is warranted.

 

INTRODUCTION

The concept of MCI both as distinct diagnostic entity & as a precursor
to Alzheimer’s (AD) suggests that instruments focused upon MCI measurement
would provide useful screening test information in primary care offices for
decisions concerning full diagnostic evaluations for AD dementia.

No single cognitive/behavioral domain can be used to differentiate persons
who will develop Alzheimer’s from those who will not. Several screening test
dimensions, when combined, significantly enhance the predictive validity of
a test battery because of variations in the initial cognitive deficits associated
with early stages of AD. Furthermore, determination of the rate of change
in at least two cognitive markers is a better means of predicting the development
of AD dementia than is a single assessment. Repeated screening test measures
are also necessary to detect changes in high functioning adults who have the
cognitive reserve to compensate for early symptoms.

Current methods of Alzheimer’s detection in primary care are costly &
often deferred until later in the disease process when interventions to delay
MCI and AD are likely to be less effective. Therefore, an effective screening
device for MCI would incorporate measures of multiple cognitive domains, measure
changes over time, & be cost efficient.

 

METHODS

Subjects

A total of 265 elderly people were recruited through senior centers &
retirement homes in 4 counties of Washington State. Exclusionary criteria
were non-English speaking, significant hand tremor, inability to sustain a
seated position for a minimum of 45 minutes, very recent surgery, cognitive
side effects of drugs, indications of recent alcohol abuse, or inadequacies
in visual acuity, hearing, or dominant hand agility.

Test Development

The CANS-MCI is a self-administered screening instrument that measures multiple
cognitive domains & has the ability to measure changes over time. Development
of the CANS-MCI tests was based upon findings of previous neuropsychological
testing research.

The usability of the CANS-MCI by elderly subjects was previously established
using the following criteria: acceptability, ease of administration, &
completion of all tests entirely by self-administration. Even study volunteers
with mild AD were able to complete the tests with minimal assistance. Moreover,
the CANS-MCI appeared to offer a way of enhancing perceptions of control over
testing & avoiding the activation of interpersonal defenses in primary
care doctors’ offices.

Both the stimulus & response characteristics of the CANS-MCI are markedly
different from traditional screening tests. The range of responses in the
CANS-MCI is limited by the touch screen modality. However, in other populations
these touch screen responses have been found to produce error rates similar
to those produced with traditional verbal responses.

Instrument Description

The CANS-MCI presents the following progression of tasks to the subjects:

Introduction

  • Presents progressively more difficult General Reaction
    Time tasks that prepare the subject for the first test.

Word/Picture Matching

  • Presents 4 pictures of objects with one word, & the
    user is instructed to touch the picture that goes with the word.

Guided Recognition-Immediate

  • User consecutively learns 5 sets of names of 4 pictures
    by touching pictures that fit into categories and then being told the name
    of the pictured object in that category.
  • User is tested after each set of 4 pictures and the set
    is re-learned if mistakes were made.
  • 20 3-button displays, each with an object name learned
    and 2 incorrect names from other categories are presented, with category-guided
    recall and re-acquisition of missed items.

Design Matching

  • Presents 8 designs paired with letters in non-alphabetical
    order, & a set of 8 letters in alphabetical order.
  • 1 of the designs appears in the middle of the screen, &
    the user is instructed to touch the letter paired with it.
  • Complexity of attention-switching
    required is increased by within & between-test interference.
  • Changes are made to the designs
    halfway through this test to present several types of interference.

Clock

  • 10 clock blank faces are presented.
  • A digital time is presented, & the user is instructed
    to first touch the hour hand position on the blank click face, then the
    minute hand position.

Stroop

  • User quickly touches buttons matching names of colors presented.
  • User is then instructed to touch buttons matching the ink
    color, not the word name, of the words “Red”, “Blue” or “Green”, presented
    one at a time in either red, blue, or green ink.

Picture Naming

  • Pictures in multiple categories are presented, each with
    4, 2-letter word beginnings, 1 naming the picture.

Guided Recognition-Delayed

  • One additional recognition test trial, with guided recall
    for missed items.

 

STATISTICAL ANALYSES

Reliability

Internal consistency: Alpha coefficient reliabilities; Test-retest:
Pearson correlations.

Validity

Concurrent: Pearson correlations with the scores on previously validated measures to provide a standard
against which the component tests could be assessed.

Diagnostic: T-tests used to analyze differences between subjects in
the lowest 10th percentile of cognitive functioning & those in the highest
90th percentile based on WMS-R LMS II scores.

Factor Analysis

Exploratory principal components factor analysis with Varimax rotation &
Kaiser normalization.

Confirmatory factor analysis to be presented at American Association of Geriatric
Psychiatry Convention, March, 2003.

 

RESULTS

Internal Consistency

Only 2 tests, Guided Recognition-Delayed (accuracy) & Guided Recognition-Guidance
Efficacy, did not meet the predetermined standard for internal consistency
(alpha >= .70) Other alpha coefficients ranged from .76-.97 (Table 1).

1 Month Test-Retest Reliability

Correlations over a 1-month period ranged from .607-.854 (Table 2). All
but 3 had scores over .70. Separately, the Guided Recognition Immediate &
Delayed accuracy tests had alphas below .70. When the immediate & delayed
recall tests were combined to form a more global memory measure, the alpha
was an acceptable .76. Guidance Efficacy had a very low test-retest alpha.
Because Guidance Efficacy scored below the cut-off criteria in both inter-item
& test-retest reliabilities, it was not included in further analyses.

Concurrent Validity

The correlations between the CANS-MCI & the previously standardized measures
were moderate but all highly significant. Correlation coefficients ranged
from .440 to .636 (p<.001) (Table 3).

Diagnostic Validation

Groups of impaired & intact memory subjects were established to assess
the degree to which the CANS-MCI was able to detect impairments in cognitive
abilities that are diagnostic of MCI or AD. Significant differences were
observed between the memory intact group & the memory-impaired group on
all CANS-MCI subtests (p<.001) (Table 4).

Factor Analysis

Results suggest a 3-factor solution that explained 63% of the total variance.
The factors were Recognition Retrieval/Language, Executive Functions &
Episodic Memory Acquisition (Table 5).

 

DISCUSSION

Reliability: The CANS-MCI demonstrates a high degree of internal consistency
& test-retest reliability. These measures of test stability are comparable
to those of the standardized comparison tests. Thus, the CANS-MCI can be
reliably used at one or multiple testing sessions. Slight improvements in
the mean scores are evident on all tests over the one-month period, probably
due to the reduction of anticipatory anxiety & establishment of positive
relationships with the participants.

Validity: Cross validation of the CANS-MCI with the WMS-R LMSI &
II, WAIS Digit Symbol & Mattis subscales demonstrates that the CANS-MCI
subtests produce meaningful score differentiation of the memory impaired &
non-memory impaired elderly. This is confirmed by an analysis based upon a
WMS-R LMS II diagnostic criterion.

Factors: The factor analysis indicated that CANS-MCI items loaded
onto 3 main factors: Recognition Retrieval/Language, Executive Functions,
& Episodic Memory Acquisition. Design Matching & Word/Picture Matching
loaded heaviest on Executive Functions but also heavily on Recognition Retrieval/Language,
reflecting the overlap of cognitive domains when recognition ability is measured
with psychomotor speed tests. Immediate & Delayed Recognition loaded
most heavily on the Episodic Memory Acquisition factor but also loaded heavily
on the Recognition Retrieval/Language factor.

 

CONCLUSION

As effective treatments for AD emerge, it will become important to identify
people in primary care office visits who have the earliest signs of the cognitive
impairments most likely to become AD. The CANS-MCI tests are reliable &
differentiate memory impaired from normal elderly, as determined by the WMS-R
LMS II. The CANS-MCI is an easily self-administered, valuable primary care
screening tool for MCI to determine whether more intensive testing for cognitive
impairment and possible dementia is warranted.

Table 1. Internal Consistency (Alpha Coefficient Reliability)

Test

#
of Items

Coefficient Alpha

General Reaction Time

10

.810

Design Matching (accuracy)

136

NA*

Clock Test (accuracy)

30

.896

Stroop Discordant Item (reaction
time)

48

.966

Guided Recognition (accuracy)

   

Immediate †
(5 Trials of 20 items)

5
(trials)

.929

Delayed (1
Trial)

20

.616

Immediate &
Delayed (combined 6 Trials)

6

.928

Guidance Efficacy

5

.568

Picture Naming (accuracy)

42

.762

Picture Naming (reaction time)

42

.793

Word/Picture Matching (reaction
time)

14

.871

Scores were only given for the items completed within the time
limit. Participants did not all answer
the same number of items so we were unable to perform reliability analyses.

† If any of the 20 correct items in a trial are not touched,
the subject receives a guided recall test on that item.

 

Table 2. Test Re-Test Reliability

Test

Time
1

Mean (SD)

Time
2

Mean (SD)

Coefficient
Alpha

General Reaction Time

0.77 (.21)

0.73 (.17)

.702

Design Matching (accuracy)

38.05 (11.41)

41.37 (8.94)

.765

Clock Test (accuracy)

30.65 (9.45)

32.86 (8.89)

.792

Stroop Discordant Item (reaction
time)

1.69 (.48)

1.61 (.52)

.794

Guided Recognition (accuracy)

     

Immediate

17.73 (2.01)

18.13 (1.86)

.681

Delayed

17.58 (2.27)

17.83 (2.25)

.607

Immediate &
Delayed (combined)

35.23 (4.21)

35.95 (3.87)

.760

Guidance Efficacy

0.86 (.15)

0.90 (.14)

.385

Picture Naming (accuracy)

31.59 (4.81)

32.00 (4.91)

.788

Picture Naming (reaction time)

6.18 (2.13)

5.90 (2.26)

.854

Word/Picture Matching (reaction
time)

2.06 (.56)

1.93 (.49)

.833

 

Table 3: Correlations of Standardized Tests with CANS-MCI Tests

Conceptual Domain

(CANS-MCI)

CANS-MCI Test

Standardized
Test

Correlation
Coefficient

P-value

Attention

General Reaction Time

Digit Symbol

-.585

<.001

Visuospatial ability

Design Matching (accuracy)

Digit Symbol

.537

<.001

Spatial relations

Clock (accuracy)

Digit Symbol

.469

<.001

Mental control

Stroop Discordant Item (latency)

Digit
Symbol

-.565

<.001

Memory acquisition

Guided Recognition- Immediate

Mattis
Memory

WMS LMS-I

.518

.540

<.001

<.001

 

Guidance Efficacy

Mattis
Memory

WMS LMS-I

.327

.352

<.001

<.001

Memory retention

Guided Recognition- Delayed

Mattis
Memory

WMS LMS-II

.447

.440

<.001

<.001

Composite memory
score

Guided Recognition- Immediate
& Delayed

Mattis
Memory

WMS LMS-I

WMS LMS-II

.486

.519

.525

<.001

<.001

<.001

Picture naming

Picture Naming (accuracy)

Mattis
Initiation

.584

<.001

 

Picture Naming (latency)

Mattis
Initiation

.616

<.001

Other fluency tests

Word/Picture Matching (latency)

Mattis
Initiation

Digit Symbol

-.496

-.636

<.001

<.001

         

 

Table 4: Diagnostic Validation using Delayed Memory Criterion

Variable

WMS-II
<10%

Mean

(SD)

WMS-II
>10%

Mean

(SD)

P-value

N

44

215

 

Demographics

     

Age

80

(8.4)

76

(8.4)

.01

Years of formal education

13

(3.1)

15

(2.7)

.02

CANS-MCI Tests

     

General Reaction Time

.91

(.28)

.73

(.17)

.000

Design Matching (accuracy)

29

(13.0)

40

(10.1)

.000

Clock (accuracy)

24

(8.8)

32

(8.9)

.000

Stroop Discordant Item (latency)

1.94

(.51)

1.64

(.45)

.000

Guided Recognition (accuracy)

     

Immediate

15

(2.7)

18

(1.4)

.000

Delayed

15

(3.2)

18

(1.8)

.000

Immediate & Delayed

(combined)

30

(6.4)

36

(2.8)

.000

Picture Naming (accuracy)

27

(5.1)

32

(4.3)

.000

Picture Naming (latlency)

8.4

(3.1)

5.8

(1.6)

.000

Word/Picture Matching (latency)

2.57

(.69)

1.95

(.46)

.000

 

Table 5. Exploratory Factor Analysis (N=132)

 
Factor
Loading

CANS-MCI Tests

Language/

Spatial Fluency

Executive Function/ Mental Control

Episodic Memory

General Reaction Time

-.294

-.741

.046

Design Matching

.497

.535

.275

Clock (accuracy)

.620

.297

.142

Stroop Discordant Item (reaction time)

-.169

-.791

-.126

Free Recognition-Immediate (accuracy)

.562

.119

.660

Free Recognition- Delayed (accuracy)

.682

-.120

.492

Picture Naming (accuracy)

.780

.308

.184

Picture Naming (reaction time)

-.825

-.242

-.231

Word/Picture Matching (reaction time)

-.543

-.568

-.167

Standardized Tests

     

WMS-R LMS-I

.192

.293

.819

WMS-R LMS-II

.211

.320

.783

Mattis Initiation

.648

.233

.361

Mattis Memory

.295

.095

.734

WAIS Digit Symbol

.377

.635

.301

Footnotes

  1. Screen, Inc. Seattle WA, USA

  2. Department of Veterans Affairs, Seattle, WA, USA

  3.  
  4. Weschler Memory Scale-Revised Logical Memory Components I & II (WMS-R LMS I & II); Mattis Dementia Rating Subscales (Mattis)-Attention, Conceptualization, Inititation, Memory; Weschler Adult Intelligence Scale, Digit Symbol Component (WAIS)

Emory Hill, Ph.D.
Founder of Screen Inc., Dr. Hill has a PhD in Clinical Psychology, State University of New York at Buffalo. Later he completed an Informatics Fellowship (post-PhD) at the VA where he studied interface design, multimedia programming, user resistance, evaluation of adaptations to new medical record systems, and the implementation of automated medical records. A trained psychologist and psychometric specialist, Emory was in private practice for nearly 20 years. Before that, he served as an Assistant Professor of Psychology at SUNY, Brockport, NY.