IntelliSpace Cognition Validation
Cognitive assessment

IntelliSpace Cognition Validation

Sign up for news and updates in cognitive assessment

Stay up-to-date and subscribe.

Join your peers, sign-up to stay informed and receive insights into healthcare innovations, straight to your inbox.

(Please be sure to check the box to receive communications from Philips)

Contact information

* This field is mandatory
*

Contact details

*
*
*

Company details

*
*
*

Philips IntelliSpace Cognition, Validated Against Traditional
Paper Neuropsychological Tests

A medical device to aid assessing cognition


As a medical device company, Philips follows the standards set by the FDA and other global organizations that govern medical devices and products. When we identified a gap in reliable digital technologies for cognitive assessment, we developed IntelliSpace Cognition (ISC) in the way we always establish our clinical offerings- through research, validation, clinical studies, and then filing with the FDA. We knew we could leverage the strong history of cognitive testing that exists with neuropsychologists while building in an added layer of insights and more accessibility that could benefit patients, neurologists, and neuropsychologists. In developing ISC, Philips invested a significant effort to validate the digital neuropsychological tests and associated algorithms as well as collect representative normative data.

Normative data collection and validation


Normative data are quantitative measures are used to compare an individual’s performance to that of a relevant population. A comparison to a representative normative baseline is essential to interpret an individual test score and ideally accounts for differences related to age, education and sex. In fact, not comparing to normative data will often result in missing subtle signs of cognitive impairment. The cognitive assessment in ISC features well-established neuropsychological tests, each with its specific normative corrections but based on the same representative group of the US population. These normed outcome measures are validated against the traditional paper versions of each test. In 2019 and 2021, Philips collected normative data and test validation data through a series of studies that were performed in accordance with the
ISO14155 standard for Good Clinical Practice.1,2 The studies involved collecting data across four different states (NY, FL, PA, CA). We administered over 1000 cognitive assessments in a group of almost 1000 healthy participants. Our participants were enrolled using stratified sampling such that distributions of sex, education level, and racial/ethnic background reflect the US Census closely.

 

For test validation we used data from these studies to compare the psychometric properties of the digital ISC tests to their paper counterparts. A group of participants in these studies completed either the digital ISC tests or the paper tests versions, either once, or twice in repeated visits about 2-3 weeks apart. The data from participants that performed both the digital ISC tests and the paper versions allow a direct comparison between the ISC tests and the traditional paper, while eliminating inter-individual differences that could otherwise obscure subtle differences.

Agreement between digital and paper tests


The most important findings from our validation work are described in a peer-reviewed scientific paper that is freely available3. We found that the ISC digital tests correlate adequately with traditional paper counterparts (see table below). While small differences between ISC and paper are to be expected, they do not influence the interpretation of ISC tests due to the integrated test specific norms that are available on the ISC digital platform. Many of the scores on ISC tests can be considered as practically equivalent to scores that would be obtained on a paper version. We also performed factor analyses to show that the ISC tests map on the same latent variables (i.e. cognitive domains) as the respective paper counterparts.

Tests on IntelliSpace Cognition compared with paper counterparts*

cognitive
Table from “Vermeent et al. 2021. Philips IntelliSpace Cognition digital test battery: equivalence and measurement invariance compared to traditional analog test versions. The Clinical Neuropsychologist”.

Automation and standardization with Artificial Intelligence

 

The very high level of automation and standardization that ISC achieves is accomplished by using cutting edge technology to adapt the way the tests are administered and scored. Algorithms in IntelliSpace Cognition that use Artificial Intelligence (AI) have been validated and perform “on par” with human neuropsychology experts (see table below).

automation
Table from “Vermeent et al. 2021. Philips IntelliSpace Cognition digital test battery: equivalence and measurement invariance compared to traditional analog test versions. The Clinical Neuropsychologist”.
An example of the use of AI to automate and standardize a test is the ISC version of the Rey Auditory Verbal Learning Test (RAVLT). Here, the word stimuli are played by the computer instead of being spoken by the human test administrator. Subsequently, the participant’s response is not written down by the administrator, but the audio is digitized, recorded, and then scored by an algorithm. The raw score is also converted into a demographically corrected normed score and into a percentile score. The benefit of this approach is that with digital technology and voice recognition the amount of manual work is drastically reduced, while maintaining the validity of the test. The algorithms do the scoring in an entirely consistent manner and we thereby eliminate inter-rater disagreement, which is a common source of error in human scoring.

Conclusion


The highly representative normative data, the similarity to the original paper tests, and the validation of algorithms against human experts form the foundation of the Philips ISC product. These aspects are important in giving the clinician confidence. In fact, when shown a video of how ISC works, 80% of neurologists said they would feel confident using IntelliSpace Cognition because Philips has collected normative data to provide healthy peer comparisons.4

More articles

References


1
 Psychometric Properties of IntelliSpace Cognition.

http://clinicaltrials.gov/ct2/show/NCT03801382
 

2 Norms Expansion and Validation for IntelliSpace Cognition.
https://clinicaltrials.gov/ct2/show/NCT04729257

 

3 Stefan Vermeent, Mandy Spaltman, Gijs van Elswijk, Justin B. Miller & Ben Schmand (2021): Philips IntelliSpace Cognition digital test battery: equivalence and measurement invariance compared to traditional analog test versions, The Clinical Neuropsychologist.

https://doi.org/10.1080/13854046.2021.1974565

 

4 Based on a 2019 Philips study of 100 neurologists in the US.

You are about to visit a Philips global content page

You are about to visit the Philips USA website.

You are about to visit a Philips global content page

You are about to visit the Philips USA website.

Our site can best be viewed with the latest version of Microsoft Edge, Google Chrome or Firefox.