Development of a measure of clinical information systems expectations and experiences

From Clinfowiki
Jump to: navigation, search

Development of a Measure of Clinical Information Systems Expectations and Experiences

Published in Med Care. 2007 Sep;45(9):884-90.


Authors

Douglas S. Wakefield, PhD Jonathon R. B. Halbesleben, PhD Marcia M. Ward, PhD Qian Qiu, MBA Jane Brokel, PhD, RN Donald Crandall, MD


From the:

  • Center for Health Care Quality, University of Missouri-Columbia, Columbia, Missouri
  • Department of Health Management and Policy

College of Nursing, University of Iowa, Iowa City, Iowa

  • Trinity Health, Novi, Michigan


Introduction

Health care organizations are undertaking ever larger investments in information technologies for the sake of improving the efficiency and quality with which patient care is delivered. Information technology use and adoption are significantly influenced by the interplay between the clinical end-users’ preliminary expectations of outcomes and their subsequent initial experiences with the technology.

It is by this reasoning that the authors identify the need for a consistent means of measuring the expectations held by clinical end users and their corresponding experiences about how these technologies change clinical care processes and work flow, particularly because no such measurement tools currently exist.

The paper documents a study conducted by the authors to validate the Information Systems Expectations and Experiences (I-SEE) survey they had designed as a tool to assess expectations and experiences regarding the impact of health information technology on clinical work processes and outcomes.


Methods

The I-SEE was developed via an iterative process among members of a study team that drew from both clinical and technical personnel with extensive experience implementing health information technology. The team then submitted the I-SEE for validation review and pilot testing by clinicians who were experienced or were currently involved in clinical information system implementation. This process involved revisions of content before the instrument was finalized.

The end product of this process was a scale survey consisting of two sections. Section I structured around 5 focus areas: provider-patient communication, inter-provider communication, inter-organizational communication, work life changes, and improved care. Section II focused on perceptions of the information system implementation strategy and perceptions of quality. The instrument was designed to be used for assessment of perceptions before and then after implementation as well.

Nursing staff served as the primary subject audience for this study. The initial validation sample included registered nurses from a large Midwestern rural referral hospital that implemented electronic medical records and computerized provider order entry systems. Nurses from 3 other hospitals were used to cross-validate the factor structure of the scale.

To assess the psychometric properties of the I-SEE scales, a variety of analyses were conducted including a basic item analysis, confirmatory factor analysis, cross-validation factor analyses, and reliability analysis.


Findings

Confirmatory factor analysis generally supported the a priori factor structure for both expectations and experiences regarding the clinical information system. The consistency of the fit to the factor models was also high across the cross-validation samples. The scales demonstrated acceptable internal consistency in all the samples.


Discussion/Conclusions

The authors conclude:

Our findings suggest that the measure of clinical information systems expectations and experiences offers a valid and reliable tool for assessing the perceived impact of new clinical technology on work process and outcomes. This instrument can be useful before and after technology implementation by assisting in the identification of staff perceptions and concerns, thus allowing for targeted interventions to address these issues.

Given such findings, they suggest that the I-SEE tool provides a mechanism for supporting further, more empirical research aimed at examining the correlation between perceptions of technology use and ensuing technology adoption—an area of research in which significant activity is lacking.

With regards leveraging the I-SEE in practice, the authors indicate that the “clearest benefit to health services clinicians and managers comes from the ability to use the measure to target specific aspects of work processes in the implementation of advanced technology and the ability to measure changes in perceptions regarding those aspects of work process.” Also, the I-SEE highlights the importance of managing expectations concerning clinical information systems so as to shape realistic expectations and consequently make for more realistic goals and therefore greater uptake of new technology.

The authors acknowledge the limitations that characterized the study:

  • Sample sizes and response rate in the initial validation sample are somewhat small
  • The study included only one occupational group, nurses. This however was by design as they “sought to minimize the possibility of confounding due to different work processes.”
  • They have reported the findings from 4 different sites, but these sites were part of a single health system. Therefore the implementation of the new clinical information system was similar across the 4 sites (e.g. same clinical information system, similar training, similar implementation strategy).


The authors recognize the need for future research to address the above limitations. In addition, future research objectives would benefit from inclusion of:

  • Research concerning the predictive validity of this measure with regard to technology adoption would allow for clear links between perceptions regarding the impact of technology and its subsequent utilization.
  • From a practical perspective, research concerning the use of the tool to guide workflow changes with technology implementation would be valuable.


Reviewer Comments

Limitations notwithstanding, the paper and associated study did raise some very valid points about assessing the relationship between the expectations and perceptions of targeted workflow changes that exist in the pre-implementation period, and the experiences and response to actual workflow changes realized in the post-implementation period. Acknowledging the need for additional research going forward, it’s worth noting that with further development the I-SEE tool has the potential to serve as a key component in the requirements analysis and design phases of a project.

And in the deployment phase could be used to determine how well the deliverables satisfy the users’ needs with regards to qualitative categories such as provider-patient communication, inter-provider communication, inter-organizational communication, work life changes, and improved care.