IntelliSpace Cognition Validation

As a medical device company, Philips follows the standards set by the FDA and other global organizations that govern medical devices and products. When we identified a gap in reliable digital technologies for cognitive assessment, we developed the IntelliSpace Cognition (ISC) product in the way we always establish our clinical offerings- through research, validation, clinical studies and then filing with the FDA. We knew we could leverage the strong history of cognitive testing that exists with neuropsychologists while building in an added layer of insights and more accessibility that could benefit patients, neurologists, and neuropsychologists. In developing ISC, Philips invested a significant effort to validate the digital neuropsychological tests and associated algorithms as well as collect accurate normative data.

Normative data are quantitative measures are used to compare an individual’s performance to that of a relevant population. A comparison to a normative baseline is essential to interpret an individual test score and ideally accounts for differences related to age, education and sex. In fact, not comparing to normative data will often result in missing subtle signs of cognitive impairment. The cognitive assessment in ISC features well-established neuropsychological tests, each with its specific normative corrections but based on the same representative group of the US population. These normed outcome measures are validated against the traditional paper versions of each test. In 2019, Philips collected the normative data and test validation data through a study that was performed in accordance with the ISO14155 standard for Good Clinical Practice.¹ It involved collecting data across four different states (NY, FL, PA, CA) from a group of 450 healthy participants enrolled using stratified sampling such that distributions of age (50-80y), sex, education level, and racial/ethnic background reflect the US Census.
 

For test validation we used data from this study to compare the psychometric properties of the digital ISC tests to their paper counterparts. Two-hundred fifty participants in the study completed either the digital ISC tests or the paper tests versions, once or twice in repeated visits about 2-3 weeks apart. Two hundred of the participants visited twice to perform both the digital ISC tests and the paper versions. The data from this specific subgroup allows a direct comparison between the ISC tests and the traditional paper while eliminating inter-individual differences that could otherwise obscure subtle differences.

The scores on the ISC digital tests correlate well with the traditional paper tests (see table)^*. While small differences between ISC and paper are to be expected, they do not influence the interpretation of ISC tests due to the integrated norms that are available on the ISC digital platform.
 

Philips recently published results concerning construct validity from a first study, based on a prototype version of the product in a peer-reviewed journal.² In this paper, structural equation modeling (SEM) was used to investigate the underlying relationships between the ISC tests with two theory-driven neuropsychological models. For both models, the outcome measures of the digital tests mapped onto the expected cognitive domains. This also suggests that the digital tests on ISC can be interpreted in the same way as traditional paper versions while offering a high level of standardization and ease-of-use. In a future paper, we will use the SEM modeling technique on the data of the aforementioned validation study from 2019, to show that the ISC tests map on the same latent variables (i.e. cognitive domains) as their respective paper counterparts.