Abstract

This chapter will examine the rationale, strategy, and methods for systematically evaluating the effectiveness of online health information systems. Evaluation research will be framed as an essential activity for designing, refining, and sustaining robust health information systems. The best health information system evaluation research programs should include: formative evaluation research activities, such as needs analysis and audience analysis, for designing responsive and appropriate systems; process evaluation research activities to assess how well health information systems work with users, primarily through use of message testing, system usage analysis, and user feedback systems; as well as summative evaluation research activities to assess the influences of the health information systems on important health outcomes, including costs and benefits. We will describe the use of multiple research methods as part of multimethodological designs for conducting health information system evaluation. These methods will include examinations of the applications of content analysis, interviews, focus groups, usability tests, cost-benefit analysis, user feedback systems, unobtrusive measures, the critical incidents method, and field experiments for evaluating health information systems.