Chapter 1: Foundation of Evaluation and Research in Healthcare

Sitaji Gurung

Learning Objectives:

• Define and discuss the differences and similarities between evaluation, research, and monitoring.
• Describe how a conceptual framework can be applied to define and evaluate healthcare systems and services.
• Discuss the three major components of the Donabedian model.
• Identify aspects of healthcare quality and explain its role in delivering healthcare services and evaluating health systems.
• Describe how accrediting organizations employ a system to evaluate healthcare services and health systems for accreditation.

Program evaluation is a systematic method for collecting, analyzing, and using data to assess the effectiveness, efficiency, and quality of programs. According to the Centers for Disease Control and Prevention (CDC), program evaluation not only helps determine whether a program is achieving its intended outcomes, but it also plays a critical role in continuous program improvement. The term “program” is defined broadly and can include policies, interventions, media campaigns, preparedness efforts, and other public health actions aimed at improving community well-being. By applying evaluation practices, health professionals can enhance accountability and drive evidence-based decision-making.

Understanding the Differences: Evaluation, Research, and Monitoring

Although evaluation, research, and monitoring are all systematic approaches to gathering and analyzing data, they serve distinct purposes in healthcare and public health contexts. Monitoring involves the routine tracking of program activities and outputs to ensure implementation is occurring as planned. It focuses on performance and operational efficiency in real time. Evaluation, by contrast, is a broader process that assesses whether a program is effective in achieving its intended outcomes and why or how those outcomes occurred. It supports decision-making and program improvement. Research, meanwhile, seeks to generate generalizable knowledge through the scientific method, often involving hypothesis testing and controlled study designs. While evaluation and monitoring focus more on program performance and management, research is oriented toward theory-building and contributing to the broader field of knowledge.

Discussion Questions

1. How can understanding the differences between monitoring, evaluation, and research help improve the effectiveness of a healthcare program?
2. In what situations might it be more appropriate to use evaluation rather than research or monitoring, and why?

Similarities Between Evaluation, Research, and Monitoring

Despite their different purposes, evaluation, research, and monitoring share several important similarities. Both research and evaluation often use comparable data collection methods and analytical techniques to examine questions, gather evidence, and interpret findings. Additionally, monitoring and evaluation (M&E) work together to assess and measure program performance. This helps organizations improve their strategies, make informed decisions, and achieve better outcomes. These overlaps show that while each process has a unique goal, they all rely on systematic data gathering and analysis to support learning, accountability, and improvement.

Discussion Questions

1. In what ways do shared data collection and analysis methods strengthen both research and evaluation processes?
2. How can integrating monitoring and evaluation (M&E) practices help organizations improve their program outcomes and decision-making?

Applying the CDC Framework for Program Evaluation

The Centers for Disease Control and Prevention’s (CDC) Framework for Program Evaluation offers a systematic approach to assessing public health programs by outlining essential steps and standards that promote both effectiveness and practical application (Figure 1). This framework is designed not only to summarize the key components of effective program evaluation, but also to integrate evaluation into the routine operations of a program. By following its steps, such as engaging stakeholders, describing the program, focusing the evaluation design, gathering evidence, and justifying conclusions, practitioners can better understand a program’s context and improve the quality of evaluation efforts. The framework also serves to clarify common misconceptions, enhance consistency, and encourage the use of findings to inform decision-making and improve program performance. Ultimately, it enables evaluators to design more useful, feasible, ethical, and accurate evaluations that support continuous improvement in public health practice.

Figure 1: CDC Program EvaluationFramework

Discussion Questions

1. Why is it important to follow a structured framework like the CDC’s when evaluating programs?
2. How can using the CDC’s evaluation steps help improve decision-making and program outcomes?

Types of Health Care Quality Measures

In research, a measure is a way to numerically represent the degree or presence of a particular attribute, whether it’s something tangible like weight or income, or more abstract like perception or satisfaction. In the context of healthcare quality, measures are used to evaluate and compare how well healthcare organizations deliver care. According to the Agency for Healthcare Research and Quality (AHRQ), these quality measures fall into three main categories based on the Donabedian model: structure, process, and outcome. Structure measures assess the resources and settings where care is delivered, such as facilities, staffing, or technology. Process measures focus on whether the steps or actions in providing care are being followed appropriately, like whether patients are receiving recommended screenings. Outcome measures look at the results of care, such as improvements in health status, patient satisfaction, or readmission rates. Together, these measures help evaluate the effectiveness, safety, and efficiency of healthcare delivery.

Understanding Structure, Process, and Outcome Measures in Health Care Quality

Health care quality measures are essential tools that help assess how well providers deliver care, and they are typically categorized into structure, process, and outcome measures. Structural measures give consumers insight into a provider’s capacity, systems, and infrastructure to deliver high-quality care. Examples include whether an organization uses electronic medical records, the number of board-certified physicians, or the provider-to-patient ratio. Process measures focus on specific actions taken by providers to promote health or manage conditions, such as the percentage of patients receiving preventive services or how well chronic conditions like diabetes are monitored and controlled. These measures are widely used in public reporting because they reflect adherence to evidence-based care guidelines. Outcome measures assess the results of health care, such as surgical mortality rates or hospital-acquired infection rates. While outcomes are often considered the “gold standard” in evaluating care, they can be influenced by many external factors. To improve accuracy, risk-adjustment methods are used to account for differences in patient populations. However, experts note that these methods are still evolving and must be refined to ensure fair and meaningful comparisons.

Discussion Questions

1. How do structural, process, and outcome measures each contribute to our understanding of health care quality, and why is it important to use all three types when evaluating a health care organization?
2. What are the benefits and challenges of using outcome measures in public health reporting, and how can risk adjustment methods help address those challenges?

The Donabedian Framework: Structure, Process, and Outcome

The Donabedian Framework is a widely used model in healthcare quality evaluation that breaks down the components of care into three essential elements: structure, process, and outcome (Figure 2). Structure refers to what a healthcare organization must have in place to provide care, such as facilities, equipment, staffing, and technology, essentially the foundational requirements. Process involves the specific actions taken during the delivery and receipt of care, such as clinical procedures, patient education, and coordination of services. Finally, outcome refers to the end results of healthcare services, including patient health status, satisfaction, and recovery rates. Together, these components help assess whether a healthcare system is effectively and efficiently providing care. This model was introduced by Avedis Donabedian, a physician and researcher renowned for his contributions to healthcare quality assessment.

Figure 2: Donabedian Framework

Discussion Questions

1. How does each component of the Donabedian Framework—structure, process, and outcome—play a role in evaluating the quality of healthcare services?
2. Can a healthcare organization achieve strong outcomes without a strong structure or process? Why or why not?

Donabedian Approach to Evaluating Health Care

The Donabedian approach is a foundational model for evaluating healthcare quality by examining three key elements: structure, process, and outcomes. This framework can be applied to assess how well a healthcare facility functions by starting with the inputs or structural components, such as the number of hospital beds, available staff (e.g., doctors, nurses, porters), the physical building, medical equipment, and funding resources. These elements provide the foundation necessary to deliver care. The next component, process, refers to how these resources are used and how care is delivered, including clinical practices, bed occupancy, prescribing behavior, referral patterns, admission procedures, and interpersonal communication during consultations. Finally, outcomes measure the impact of these processes on patient health and service efficiency, including indicators like mortality, morbidity, complication rates, disability, length of hospital stay, discharge rates, and quality of life. This approach allows healthcare organizations to identify areas for improvement, ensuring that inputs are effectively utilized, care is delivered efficiently, and patients experience meaningful health outcomes.

Discussion Questions

1. How can evaluating inputs, processes, and outcomes help healthcare organizations identify specific areas for improvement?
2. Which of the three components—structure, process, or outcomes—do you think has the greatest impact on patient care, and why?

References

Centers for Disease Control and Prevention (n.d.). CDC Approach to Program Evaluation. U.S. Department of Health and Human Services.

Centers for Disease Control and Prevention (n.d.). CDC Program Evaluation Framework. U.S. Department of Health and Human Services.

Agency for Healthcare Research and Quality (AHRQ). (2015, July). Types of health care quality measures. U.S. Department of Health and Human Services.

Hamilton, K., Howell, D., & Jordan, M. (2021). The fundamentals of healthcare administration: Navigating challenges and coordinating care (Chapter 4: Emerging trends in healthcare organizations and their managerial implications). University of North Georgia Press.

Donabedian, A. (2005). Evaluating the quality of medical care. The Milbank Quarterly, 83(4), 691–729.