IntelliSpace Cognition Validation

A medical device to aid assessing cognition

As a medical device company, Philips follows the standards set by the FDA and other global organizations that govern medical devices and products. When we identified a gap in reliable digital technologies for cognitive assessment, we developed IntelliSpace Cognition (ISC) in the way we always establish our clinical offerings- through research, validation, clinical studies, and then filing with the FDA. We knew we could leverage the strong history of cognitive testing that exists with neuropsychologists while building in an added layer of insights and more accessibility that could benefit patients, neurologists, and neuropsychologists. In developing ISC, Philips invested a significant effort to validate the digital neuropsychological tests and associated algorithms as well as collect representative normative data.

Normative data collection and validation

Normative data are quantitative measures are used to compare an individual’s performance to that of a relevant population. A comparison to a representative normative baseline is essential to interpret an individual test score and ideally accounts for differences related to age, education and sex. In fact, not comparing to normative data will often result in missing subtle signs of cognitive impairment. The cognitive assessment in ISC features well-established neuropsychological tests, each with its specific normative corrections but based on the same representative group of the US population. These normed outcome measures are validated against the traditional paper versions of each test. In 2019 and 2021, Philips collected normative data and test validation data through a series of studies that were performed in accordance with the
ISO14155 standard for Good Clinical Practice.1,2 The studies involved collecting data across four different states (NY, FL, PA, CA). We administered over 1000 cognitive assessments in a group of almost 1000 healthy participants. Our participants were enrolled using stratified sampling such that distributions of sex, education level, and racial/ethnic background reflect the US Census closely.

For test validation we used data from these studies to compare the psychometric properties of the digital ISC tests to their paper counterparts. A group of participants in these studies completed either the digital ISC tests or the paper tests versions, either once, or twice in repeated visits about 2-3 weeks apart. The data from participants that performed both the digital ISC tests and the paper versions allow a direct comparison between the ISC tests and the traditional paper, while eliminating inter-individual differences that could otherwise obscure subtle differences.

Agreement between digital and paper tests

The most important findings from our validation work are described in a peer-reviewed scientific paper that is freely available3. We found that the ISC digital tests correlate adequately with traditional paper counterparts (see table below). While small differences between ISC and paper are to be expected, they do not influence the interpretation of ISC tests due to the integrated test specific norms that are available on the ISC digital platform. Many of the scores on ISC tests can be considered as practically equivalent to scores that would be obtained on a paper version. We also performed factor analyses to show that the ISC tests map on the same latent variables (i.e. cognitive domains) as the respective paper counterparts.

Tests on IntelliSpace Cognition compared with paper counterparts*