Too often, interventional programs are initiated and implemented at great expense with no thought of how to measure the success (or failure) of these programs. Success can take many forms, and the outcome measurements utilized must be identified as part of the planning process. At all levels, from the DNP Project to NIH grants, the results of these projects and programs are of little worth without measurable, pre-planned outcomes.
According to Gordis, ?outcomes research is used to denote studies comparing the effects of two or more interventions or modalities, such as treatments, forms of health care organizations, or types and extent of insurance coverage and provider reimbursement on health or economic outcomes. Endpoints can include morbidity and mortality as well as measures of quality of life, functional status, and patient perceptions of their health status, including symptom recognition and patient satisfaction? (2013, p. 313). The American Nurses Association defines outcome measurements as collecting and analyzing data using predetermined outcome indicators for the purpose of making decisions about healthcare? (ANA, 2004).
Healthy People (HP) 2020 has identified 12 leading health indicators to communicate high-priority health issues with respective recommended interventions and proposed outcomes. These areas include access to health services; clinical preventive services; environmental quality; injury and violence; maternal, infant, and child health; mental health; nutrition; physical activity and obesity; oral health; reproductive and sexual health; substance abuse; and tobacco use. Each of the 12 topics has indicators that will be tracked, measured, and reported on throughout the decade. HP is soliciting input from healthcare providers for innovative, cost-effective, sustainable programs to address these high-priority areas.
Regardless of what definition or combination of definitions one espouses, the necessity of incorporating ?outcome measurements? into practice cannot be minimized. Too often, interventional programs are initiated and implemented at great expense, with no thought of how to measure the success or failure of these programs. Success can take many forms (as indicated above) and the outcome measurements utilized must be identified as part of the planning process. At all levels, from small projects to NIH research, the results of these projects and programs are of little worth without measurable, pre-planned outcomes.
There is a variety of frameworks that are used in nursing and healthcare to evaluate outcomes.
1. Clinical Value Compass utilizing four dimensions:
? Clinical?disease-specific outcomes
? Functional?ability to participate in activities of daily living, overall well-being
? Cost?number of encounters, length of stay, finances and resources
? Satisfaction?patient and family satisfaction (Nelson et al., 1995; Nelson, Mohr, Batalden, & Plume, 1996; Oermann & Floyd, 2001).
2. Structure, Process, and Outcomes
Process refers to recognized components of what has been identified as good care. Identifying what constitutes good care is a decision often made by an expert panel. A clinical or healthcare provider can be assessed by reviewing relevant records or by direct observation to determine to what extent the care provided meets established and accepted criteria.
What are disadvantages of using process as a measurement of good” care?
The inherent flaws of process measurement have led to the acceptance and utilization of outcomes measurements. The term outcomes denotes whether the patient actually benefits from the care provided. In the past, most outcomes were measured easily using mortality and morbidity. There has been a shift from morbidity and mortality measurements in recent years to more sensitive measures such as quality of life, independence, pain levels, ability to ambulate, and so forth.
When the literature evaluates the benefit that a patient receives from a medical intervention, three words are employed: efficacy, effectiveness, and efficiency.
Efficacy refers to the agents ability to produce results in a perfectly controlled environment.
Example: Some randomized clinical trials pay carefully screened volunteers to spend several days in a clinic-type environment where they receive medications administered by the staff. Their diet and all other activities are supervised and controlled. Therefore, there is a high level of confidence that any results or outcomes are due to the medications that were administered by the staff.
Effectiveness is the result of the agent when utilized in a real-life situation. Is it effective?
Example: Iron tablets may decrease anemia in pregnant women, but many patients will not take it because of the major side effects, which include constipation. Many patients may benefit from medication, but for a variety of reasons, they will not take it. Although the medication is efficacious, if patients refuse it, no benefit will result.
Efficiency considers factors that may make one effective option preferable over another.
Example: The cost-benefit ratio of a particular treatment may be considered. Is it possible to achieve the same benefits in a cheaper, better way? Costs can include financial, personal inconvenience, side effects, social acceptance, pain, time, and others.
Efficacy is determined first. Once an intervention is proven efficacious in a controlled environment, it is evaluated for effectiveness in real-life situations.
Effectiveness is determined next. Only effective interventions are evaluated for efficiency.
Efficiency is determined after efficacy and effectiveness are established. If an intervention is inefficient, it may be impractical to implement.
Card 1 of 3
View each term then click Flip Card” to see the definition.
According to Gordis (2009), factors that must be considered when developing outcome measures are
1. the measure must be clearly quantifiable;
2. the measure of outcome should be relatively easy to define and diagnose;
3. the measures selected should lend themselves to standardization for study purposes and replication; and
4. the population served and the comparison population must be at risk for the same condition being evaluated.
Example: Possible endpoints for measuring the success of a vaccine program are
1. the number (or proportion) of people immunized;
2. the number (or proportion) of people at high risk who are immunized;
3. the number (or proportion) of people immunized who show serologic response;
4. the number (or proportion) of people immunized, who are later exposed and the disease does not develop; and
5. the number or proportion of people immunized later, are exposed, and in whom clinical or subclinical disease does not develop.
Example: Possible endpoints for measuring the success of a throat culture program include
1. the number of cultures taken (symptomatic or asymptomatic);
2. the number (or proportion) of cultures positive for streptococcal infection;
3. the number (or proportion) of people with positive cultures for whom medical care is obtained;
4. the number (or proportion) of people with positive cultures for whom proper treatment is prescribed and taken;
5. the number (or proportion) of positive cultures followed by a relapse; and
6. the number (or proportion) of positive cultures followed by rheumatic fever.
Examples of Outcome Measures
The National Health Service (NHS) in England began the Patient Reported Outcomes Measures (PROM) in 2009 (Barham & Devlin, 2011). Up until this time, most outcomes were based on the number and speed of services delivered and any associated adverse effects. This is the first time that patients self-perceived quality of health was being used as an outcome. It is a self-reported measure of how patients perceive their health both before and after surgery. Patients assess and report their health periodically. These appraisals are compared to measured change in health that occurred. The information that is gathered
1. helps patients make informed decisions;
2. allows clinicians to monitor in an organized fashion;
3. improves quality;
4. informs insurance sources as to which service to prioritize; and
5. rewards good performance by providers.
In addition, this type of research offers the nurse a variety of utilization techniques. PROM can serve as a framework to develop an evidence base for nursing practice. It can also facilitate the measurement of the effectiveness of holistic nursing interventions, and it provides an opportunity to develop our own measurements to support our interventions. Evidence-based practice standards will need to measure nursing interventions to ensure that information is meaningful.
An observational outcomes study by Romley, Anupam, and Goldman (2011) was conducted at 208 California hospitals from 1999 to 2008. Approximately 2.5 million patients who had any of six major medical conditions associated with inpatient quality indicators were evaluated for mortality rates. The end-of-life hospital spending was found to have a negative correlation to mortality (i.e., the higher the spending, the lower the inpatient mortality rate). There was no variation by region or size of hospital.
The U.S. Preventative Services Task Force (USPSTF) has based its recommendations on an evidence-based model of clinical prevention (Leipzig, Whitlock, Wolff, Barton, Michael, Harris, et. al., 2010). However, patients with multifactorial serious illness may complicate outcomes research. Most outcomes measurements are based on one disease or condition and the associated outcomes. Elderly populations are increasingly likely to have comorbidities that are not considered; therefore, the results of most outcomes measurements are not easily summarized for systematic review and the development of recommendations.
Individuals vs. Groups
Outcomes are most often measured at the individual level with no consideration of the interaction and dynamics between patients in the same groups.
Do interventions that work with individuals transfer successfully to groups? Or are there intervening variables caused by group dynamics that affect the successful transfer?
The literature is unclear about which intervention is more effective and efficient at improving health outcomes (individual or group). There is a variety of group factors that can create an influence, including leadership style, the characteristics of the individual participants, and the multiple interactions that take place within a group context. Hoddinott, Allan, Avenell, and Britten (2010) propose an in-depth framework to be utilized as an initial step toward creating interventions for groups.
Another consideration for outcomes measurement has been discussed by McLeroy, Bibeau, Steckler, and Glanz (1988). They believe that the level at which the intervention occurs needs to be evaluated because it will influence the outcome measurements. They theorize that the macro, meso, or micro levels each have variables that can affect individual and group outcomes.
There is no doubt that healthcare providers need outcome measurements to evaluate healthcare interventions for a myriad of reasons. As we search for new approaches to address the gaps in our healthcare system, outcomes will play an increasingly important role in the decision-making process regarding which interventions best serve our patients and society in a cost-effective manner. Unfortunately, the assumption that individual healthcare outcomes can be extrapolated to group healthcare outcomes has yet to be confirmed. Practitioners and policy makers would be in a better position to discern which interventions are appropriate for groups based on outcomes if they had an evidence base to support group interventions.
American Nurses Association. (2004). Nursing: Scope and standards of practice. Washington, DC: Author.
Barham, L. & Devlin, N. (2011). Patient-reported outcome measures: Implications for nursing. Nursing Standard, 25(18), 42?45.
Gordis, L. (2009). Epidemiology (4th ed.). Philadelphia, PA: Elsevier Saunders.
Hoddinott, P., Allan, K., Avenell, A., & Britten, J. (2010). Group interventions to improve health outcomes. BMC Public Health, 10, 800. doi: 10.1186/1471-2458-10-800.
Leipzig, R. M., Whitlock, E. P., Wolff, T. A., Barton, M. B., Michael, Y. L., Harris, R., et. al. (2010). Making prevention recommendations relevant for an aging population. Annals of Internal Medicine, 153(12), 843?844.
McLeroy, K. R., Bibeau, D., Steckler, A., & Glanz, K. (1988). An ecological perspective health promotion programs. Health Education & Behaviors, 15(4) 351?377. doi:10.1016/SO149-7189(97)00016-5.
Nelson, E.C., Batalden, P.B., Plume, S. K., Miheve, N.T. & Swartz, W. G. (1995). Report cards or instrument panels: Who needs what? The Joint Commission Journal on Quality Improvement, 21(4), 155?166.
Nelson, E. C., Morh, J. J., Batalden, P. B., & Plume, S. K. (1996). Improving healthcare, part 1: The clinical value compass. Joint Commission Journal on Quality Improvement, 22(4), 243?58.
Romley, J. A., Anupam, B. J., & Goldman, D. P. (2011). Hospital spending and inpatient mortality: Evidence from California. Annals of Internal Medicine, 154(3), 160?167.