Using Epidemiology to Evaluate Health Services Paper

Using Epidemiology to Evaluate Health Services Paper Using Epidemiology to Evaluate Health Services Paper Permalink: https://nursingpaperessays.com/ using-epidemiolo…h-services-paper / D-From Gordis TO ASSIGNMENTS (NOT DISCUSSION): Gordis-Review all problems and answers in Ch 16, 17, 18. No problems to submit. Gordis assignment — instead of problems, Post to Assignments (NOT Discussion).(1) From Gordis Ch. 17, 18, & 19 (each), on review of each of these 3 chapters, post and discuss any 1 epidemiology concept in each listed chapter that you better understand see highlighted concepts in each chapter (2) Include your possible use of your example (from each chapter) in the future practice (Primary Care at a community health clinic).(3) Offer any concepts you plan to further explore. Using Epidemiology to Evaluate Health Services Paper Post as: CH 17-post & discuss any 1 epi concept in that chapter with 1 future application of either concept.CH 18-post & discuss any 1 epi concept in that chapter with 1 future application of either concept.CH 19-post & discuss any 1 epi concept in that chapter with 1 future application of either concept. Gordis CHAPTER 17 Using Epidemiology to Evaluate Health Services Keywords measures of process and outcome; efficacy, effectiveness, and efficiency; outcomes research; avoidable mortality; Healthy People 2020 health indicators LEARNING OBJECTIVES To distinguish measures of process from measures of outcome, and to discuss some commonly used measures of outcome in health services research. To define efficacy, effectiveness, and efficiency in the context of health services. To compare and contrast epidemiologic studies of disease etiology with epidemiologic studies evaluating health services. To discuss outcomes research in the context of ecologic data, and to present some potential biases in epidemiologic studies that emerge when evaluating health services using group-level data. To describe some possible study designs that can be used to evaluate health services using individual-level data, including randomized and nonrandomized designs. Perhaps the earliest example of an evaluation is the description of creation given in the book of Genesis 1:1–4, which is shown in the original Hebrew in Fig. 17.1. Translated, with the addition of a few subheadings, it reads as follows BASELINE DATA In the beginning God created the heaven and the earth. And the earth was unformed and void and darkness was on the face of the deep. IMPLEMENTATION OF THE PROGRAM And God said, “Let there be light.” And there was light. EVALUATION OF THE PROGRAM And God saw the light, that it was good. FURTHER PROGRAM ACTIVITIES And God divided the light from the darkness. This excerpt includes all of the basic components of the process of evaluation: baseline data, implementation of the program, evaluation of the program, and implementation of new program activities on the basis of the results of the evaluation. However, two problems arise in this description. First, we are not given the precise criteria that were used to determine whether or how the program was “good”; we are told only that God saw that it was good (which, in hindsight, may be sufficient). Second, this evaluation exemplifies a frequently observed problem: the program director is assessing his own program. Both conscious and subconscious biases can arise in evaluation. Furthermore, even if the program director administers the program superbly, he or she may not necessarily have the specific skills that are needed to conduct a methodologically rigorous evaluation of the program. Dr. Wade Hampton Frost, a leader in epidemiology in the early part of the 20th century, addressed the use of epidemiology in the evaluation of public health programs in a presentation to the American Public Health Association in 1925. 1 He wrote, in part, as follows: The health officer occupies the position of an agent to whom the public entrusts certain of its resources in public money and cooperation, to be so invested that they may yield the best returns in health; and in discharging the responsibilities of this position he is expected to follow the same general principles of procedure as would be a fiscal agent under like circumstances. … Since his capital comes entirely from the public, it is reasonable to expect that he will be prepared to explain to the public his reasons for making each investment, and to give them some estimate of the returns which he expects. Nor can he consider it unreasonable if the public should wish to have an accounting from time to time, to know what returns are actually being received and how they check with the advance estimates which he has given them. Certainly any fiscal agent would expect to have his judgment thus checked and to gain or lose his clients’ confidence in proportion as his estimates were verified or not. However, as to such accounting, the health officer finds himself in a difficult and possibly embarrassing position, for while he may give a fairly exact statement of how much money and effort he has put into each of his several activities, he can rarely if ever give an equally exact or simple accounting of the returns from these investments considered separately and individually. This, to be sure, is not altogether his fault. It is due primarily to the character of the dividends from public health endeavor, and the manner in which they are distributed. They are not received in separate installments of a uniform currency, each docketed as to its source and recorded as received; but come irregularly from day to day, distributed to unidentified individuals throughout the community, who are not individually conscious of having received them. They are positive benefits in added life and improved health, but the only record ordinarily kept in morbidity and mortality statistics is the partial and negative record of death and of illness from certain clearly defined types of disease, chiefly the more acute communicable diseases, which constitute only a fraction of the total morbidity. 1 Using Epidemiology to Evaluate Health Services Paper Dr. Charles V. Chapin commented on Frost’s presentation: Dr. Frost’s earnest demand that the procedures of preventive medicine be placed on a firm scientific basis is well timed. Indeed, it would have been opportune at any time during the past 40 years and, it is to be feared, will be equally needed for 40 years to come. 2 Chapin clearly underestimated the number of years; the need remains as critical today, some 90+ years later, as it was in 1925. Studies of Process and Outcome Avedis Donabedian is widely regarded as the author of the seminal work on creating a framework of examining health services in relation to the quality of care. He identified three important factors simultaneously at play: (1) structure, (2) process, and (3) outcome. Structure relates to the physical locations where care is provided, the personnel, equipment, and financing. We will restrict our discussion here to the remaining two components, process and outcome. Studies of Process At the outset, we should distinguish between process and outcome studies. Process means that we decide what constitutes the components of good care, services, or preventive actions. Such a decision may first be made by an expert panel. We can then assess a clinic or health care provider, by reviewing relevant records or by direct observation, and determine to what extent the care provided meets established and accepted criteria. For example, in primary care we can determine what percentage of patients have had their blood pressure measured. The problem with such process measures is that they do not indicate whether the patient is better off; for example, monitoring blood pressure does not ensure that the patient’s blood pressure is under control or that the patient will consistently take antihypertensive medications if they are prescribed. Second, because process assessments are often based on expert opinion, the criteria used in process evaluations may change over time as expert opinion changes. For example, in the 1940s, the accepted standard of care for premature infants required that such infants be placed in 100% oxygen. Incubators were monitored to be sure that such levels were maintained. However, when research demonstrated that high oxygen concentration played a major role in producing retrolental fibroplasia—a form of blindness in children who had been born prematurely—high concentrations of oxygen were subsequently deemed unacceptable. Studies of Outcome Given the limitations of process studies, the remainder of this chapter focuses on outcome measures. Outcome denotes whether or not a patient (or a community at large) benefits from the medical care provided. Health outcomes are frequently considered the domain of epidemiology. Although such measures have traditionally been mortality and morbidity, interest in outcomes research in recent years has expanded the measures of interest to include patient satisfaction, quality of life, degree of dependence and disability, and similar measures. Efficacy, Effectiveness, and Efficiency Three terms that are often encountered in the literature dealing with evaluation of health services are efficacy, effectiveness, and efficiency. These terms are often used in association with the findings from randomized trials. Efficacy Does the agent or intervention “work” under ideal “laboratory” conditions? We test a new drug in a group of patients who have agreed to be hospitalized and who are observed as they take their therapy. Or a vaccine is tested in a group of consenting subjects. Thus, efficacy is a measure in a situation in which all conditions are controlled to maximize the effect of the agent. Generally, “ideal” conditions are those that occur in testing a new agent of intervention using a randomized trial. Effectiveness If we administer the agent in a “real-life” situation, is it effective? For example, when a vaccine is tested in a community, many individuals may not come in to be vaccinated. Or, an oral medication may have such an undesirable taste that no one will take it (so that it will prove ineffective), despite the fact that under controlled conditions, when compliance was ensured, the drug was shown to be efficacious. Efficiency If an agent is shown to be effective, what is the cost–benefit ratio? Is it possible to achieve our goals in a less expensive and better way? Cost includes not only money, but also discomfort, pain, absenteeism, disability, and social stigma. If a health care measure has not been demonstrated to be effective, there is little point looking at efficiency, for if it is not effective, the least expensive alternative is not to use it at all. At times, of course, political and societal pressures may drive a program even if it is not effective (an often-cited example is DARE—Drug Abuse Resistance Education, which has never been shown to have an impact on adolescent and young adult drug use). However, this chapter will focus only on the science of evaluation and specifically on the issue of effectiveness in evaluating health services. Measures of Outcome If efficacy of a measure has been demonstrated—that is, if the methods of prevention and intervention that are of interest have been shown to work—we can then turn to evaluating effectiveness. What guidelines should we use in selecting an appropriate outcome measure to serve as an index of effectiveness? First, the measure must be clearly quantifiable; that is, we must be able to express its effect in quantitative terms. Second, the measure of outcome should be relatively easy to define and diagnose. If the measure is to be used in a population study, we would certainly not want to depend on an invasive procedure for assessing any benefits. Third, the measure selected should lend itself to standardization for study purposes. Fourth, the population served (and the comparison population) must be at risk for the same condition for which an intervention is being evaluated. For example, it would obviously make little sense to test the effectiveness of a sickle cell screening program in a white population in North America (as sickle cell disease primarily affects African Americans). The type of health outcome end point that we select clearly should depend on the question that we are asking. Although this may seem self-evident, it is not always immediately apparent. Box 17.1 shows possible end points in evaluating the effectiveness of a vaccine program. Whatever outcome we select should be explicitly stated so that others reading the report of our findings will be able to make their own judgments regarding the appropriateness of the measure selected and the quality of the data. Whether the measure we have selected is indeed an appropriate one depends on clinical and public health aspects of the disease or health condition in question. Box 17.1 Some Possible End Points for Measuring the Success of a Vaccine Program Number (or proportion) of people immunized Number (or proportion) of people at (high) risk who are immunized Number (or proportion) of people immunized who show serologic response Number (or proportion) of people immunized and later exposed in whom clinical disease does not develop Number (or proportion) of people immunized and later exposed in whom clinical or subclinical disease does not develop Box 17.2 shows possible choices of measures for assessing the effectiveness of a throat culture program in children. Measures of volume of services provided, numbers of cultures taken, and number of clinic visits have been traditionally used because they are relatively easy to count and are helpful in justifying requests for budgetary increases for the program in the following year. However, such measures are all process measures and tell us nothing about the effectiveness of an intervention. We therefore move to other possibilities listed in this box. Again, the most appropriate measures should depend on the question being asked. The question must be specific. It is not enough just to ask how good the program is. Box 17.2 Some Possible End Points for Measuring Success of a Throat Culture Program Number of cultures taken (symptomatic or asymptomatic) Number (or proportion) of cultures positive for streptococcal infection Number (or proportion) of persons with positive cultures for whom medical care is obtained Number (or proportion) of persons with positive cultures for whom proper treatment is prescribed and taken Number (or proportion) of positive cultures followed by a relapse Number (or proportion) of positive cultures followed by rheumatic fever Comparing Epidemiologic Studies of Disease Etiology and Epidemiologic Research Evaluating Effectiveness of Health Services In classic epidemiologic studies of disease etiology, we examine the possible relationship between a putative cause (the independent variable or “exposure”) and an adverse health effect or effects (the dependent variable or “outcome”). In doing so, we take into account other factors, including health care, that may modify the relationship or confound it (Fig. 17.2A). In health services research, we focus on the health service as the independent variable (the “exposure”), with a reduction in adverse health effects as the anticipated outcome (dependent variable) if the modality of care is effective. In this situation, environmental and other factors that may influence the relationship are also taken into account (see Fig. 17.2B). Thus, both etiologic epidemiologic research and health services research address the possible relationship between an independent variable and a dependent variable, and the influence of other factors on the relationship. Therefore, it is not surprising that many of the study designs discussed are common to both epidemiologic and health services research, as are the methodologic problems and potential biases that may characterize these types of studies. Using Epidemiology to Evaluate Health Services Paper FIG. 17.2 (A) Classic epidemiologic research into etiology, taking into account the possible influence of other factors, including health care. (B) Classic health services research into effectiveness, taking into account the possible influence of environmental and other factors. Evaluation Using Group Data Regularly available data, such as mortality data and hospitalization data, are often used in evaluation studies. Such data can be obtained from different sources, and such sources may differ in important ways. For example, Fig. 17.3 shows the changes in the estimated proportion of the US population with influenza-like illness (ILI) over time—trends—using three different data sources: sentinel surveillance sites overseen by the Centers for Disease Control and Prevention (CDC), Google Flu Trends, and Flu Near You. 3 FIG. 17.3 Estimated proportion of US population with influenza-like illness January 2011–13. CDC, Centers for Disease Control and Prevention. (From Butler D. When Google got flu wrong. Nature. 2013;494:155–156.) Although the trends are fairly similar in this time period, we can see that Google Flu Trends estimated a higher proportion of the US population with ILI toward the end of 2012, nearly twice as high as the CDC estimates. This is potentially attributed to the varying methodology of data collection of each data source. The CDC generates its data from over 2,700 health care centers that capture over 30 million patient visits each year. Google Flu Trends uses data mining and modeling methodology generated from the flu-related search terms entered in Google’s search engine. Flu Near You uses data entered by internet users volunteering information, not necessarily physicians, to report on a weekly basis whether they, or their family members, have ILI symptoms. It is possible that not all individuals who develop ILI symptoms will seek medical care, and hence are not captured by the CDC data, but they may perform a Google search for ways to alleviate ILI symptoms, for example. Since Flu Near You solely depends on voluntary self-report of ILI symptoms it might well underestimate prevalence. In a recent flu season, New York State Governor Andrew M. Cuomo declared a Public Health Emergency in response to a severe flu season. It was suggested that this might have prompted numerous searches on Google by individuals who are not actually suffering from ILI symptoms, which in turn could have triggered the spike that we see in the figure. Outcomes Research The term outcomes research has been increasingly used to denote studies comparing the effects of two or more health care interventions or modalities—such as treatments, forms of health care organization, or type and extent of insurance coverage and provider reimbursement—on health or economic outcomes. The health end points may include morbidity and mortality as well as measures of quality of life, functional status, and patient perceptions of their health status, including symptom recognition and patient-reported satisfaction. Economic measures may reflect direct or indirect costs, and can include hospitalization rates, rehospitalization for the same condition within 30 days of discharge, outpatient and emergency room visits, lost days of work, child care, and days of restricted activity. Consequently, epidemiology is one of several disciplines needed in outcomes research. Outcomes research often uses data from large data sets that were derived from large populations. Although in recent years some of the large data sets have been developed from cohorts that were originally set up for different research purposes, many of the data sets used were often originally initiated for administrative or fiscal purposes, rather than for any research goals. Often several large data sets, each having information on different variables, may be combined or linked (resulting in “meta-data”) in order to have sufficient sample size to explore a question of interest. With the advent of the electronic medical record (EMR), patient care data are increasingly available to the epidemiology and health services research communities. The purpose of the EMR is to provide health care providers all of the information pertaining to individual patients—findings from office visits, utilization of preventive services, prescribed medications, procedures, radiologic findings, laboratory test results—continuously over time (i.e., prospectively). However, the purpose of the EMR is not to serve as a research base but to direct patient care. Harnessing the EMR to evaluate health services research questions has great promise, but to date it has proven difficult to use and the methods to maximize its potential are still being developed and tested in the field. The advantages of using large data sets (sometimes referred to as “big data”) are that the data refer to real-world populations, and the issue of “representativeness” or “generalizability” is minimized. In addition, since the data sets exist at the time the research is initiated, analysis can generally be completed and results generated relatively rapidly. Moreover, given the large data sets used, sample size is not usually a problem except when smaller subgroups are examined. Given these considerations, the costs of using existing data sets are generally lower than the costs of primary data collection. The disadvantages are that, since the data were often initially gathered for fiscal patient care and administrative purposes, they may not be well suited for research purposes and for answering the specific research question addressed in the study. Even when the data were originally gathered for research, our knowledge of the area may now be more complete and new research questions may have arisen that were not even conceived of when the original data collection was initiated. In general, data may be incomplete. Data on the independent and dependent variables may be very limited. Data may be missing on clinical details including disease severity and on the details of interventions, and diagnostic coding may be inconsistent across facilities and within facilities over time. Data relating to possible confounders may be inadequate or absent since the research now being conducted was often not even possible when the data were originally generated. Because certain variables that today are considered relevant and important were not included in the original data set, investigators may at times create surrogate variables for the missing variables, using certain variables that are included in the data set but that may not directly reflect the variable of interest. However, such surrogate variables vary in the extent to which they are an adequate measure of the missing variable of interest. For all these reasons, the validity of the conclusions reached may therefore be in doubt. Using Epidemiology to Evaluate Health Services Paper Another important problem that may arise with large data sets is that because the necessary variables may be absent in the available data set, the investigator may consciously or subconsciously change from the question he or she had originally wanted to address to a question that is of less interest, but for which the variables that are needed for conducting the study are present in the data set. Thus, rather than the investigator deciding what research question should be addressed, the data set itself may end up determining what questions are asked in the study. Finally, using large data sets, investigators become progressively more removed from the individuals being studied. Over the years, direct interviews and reviews of patient records have tended to be replaced by large computerized databases. Using these sources of data, many personal characteristics of the subjects are never explored and their relevance to the questions being asked is virtually never assessed. One area in which existing sources of data are often used in evaluation studies is prenatal care. The problems discussed earlier are exemplified in the use of birth certificates. These documents are often used because they are easily accessible and provide certain medical care data, such as the trimester in which prenatal care was begun. However, birth certificates for women with high-risk pregnancies have missing data more often than those for women with low-risk pregnancies. The quality of the data provided on birth certificates also may differ regionally and internationally, and may complicate any comparisons that are made. An example of outcomes research using large data sets is a study by Ikuta et?al. of Medicare beneficiaries in the United States. 4 Since Medicare health coverage is provided to virtually all elderly (ages 65 years and older) individuals in the United States, it is assumed that if a study population is limited to those who have Medicare coverage, financial obstacles to care and other variables such as age, gender, or racial/ethnic subpopulations are held constant among different groups. However, wide disparities still remain between blacks and whites in utilizing many Medicare services. The authors studied the national trends in the use of pulmonary artery catheterization (PAC) among Medicare beneficiaries during the period 1999–2013. 4 PAC is a procedure by which a tube is inserted in one of the large veins in the body, and then threaded through the heart to be ultimately placed in the pulmonary artery. This procedure used to be indicated as part of routine management of heart failure and sepsis-related acute respiratory distress syndrome, among many others. However, given the rising evidence that PAC did not improve patient outcomes, the clinical practice guidelines of the American College of Cardiology and the Society of Critical Care Medicine now recommends against the routine use of PAC. The authors studied inpatient claims data from the Centers for Medicare and Medicaid Services from 1999 to 2013 and estimated the rate of use of a PAC per 1,000 admissions, 30-day mortality, and length of stay. They found a statistically significant 67.8% relative reduction in PAC use (6.28 per 1,000 admissions in 1999 to 2.02 per 1,000 admissions in 2013), in addition to year-to-year reductions in in-hospital mortality, 30-day mortality, and length of stay. However, the findings also showed that such rates varied substantially by gender (Fig. 17.4), race (Fig. 17.5), and age (Fig. 17.6). These results showed the added benefits in restricting the use of PAC in some patients. In the meantime, the authors admitted the limitations in the use of administrative data sets and the inability to generalize to younger and uninsured individuals. FIG. 17.4 Pulmonary artery catheter use rate per 1,000 admissions by gender between 1999 and 2013. (Modified from Ikuta K, Wang Y, Robinson A, et?al. National trends in use and outcomes of pulmonary artery catheters among medicare beneficiaries, 1999–2013. JAMA Cardiol. 2017;2:908–913.) FIG. 17.5 Pulmonary artery catheter use rate per 1,000 admissions by race between 1999 and 2013. (Modified from Ikuta K, Wang Y, Robinson A, et?al. National trends in use and outcomes of pulmonary artery catheters among medicare beneficiaries, 1999–2013. JAMA Cardiol. 2017;2:908–913.) FIG. 17.6 Pulmonary artery catheter use rate per 1,000 admissions by age groups between 1999 and 2013. (Modified from Ikuta K, Wang Y, Robinson A, et?al. National trends in use and outcomes of pulmonary artery catheters among medicare beneficiaries, 1999–2013. JAMA Cardiol. 2017;2:908–913.) Potential Biases in Evaluating Health Services Using Group Data Studies evaluating health services using group data are susceptible to many of the biases that characterize etiologic studies, as discussed in Chapter 15. In addition, certain biases are particularly relevant for specific research areas and topics, and may be important depending on the specific epidemiologic design selected. For example, studies of the relationship of prenatal care to birth outcomes are prone to several important potential biases. In such studies, the question often addressed is whether prenatal care, as measured by the absolute number of prenatal visits, reduces the risk of prematurity and low birth weight. Several potential biases may be introduced into this type of analysis. For example, other things being equal, a woman who delivers prematurely will have fewer prenatal visits (i.e., the pregnancy was shorter so that there was less time in which it was possible for her to “be at risk” for prenatal visits). The result would be an artefactual relationship between fewer prenatal visits and prematurity, only because the gestation was shorter. However, bias can also operate in the other direction. A woman who begins prenatal care in the last trimester of pregnancy will likely not have an early premature delivery, as she has already carried the pregnancy into the last trimester. This would lead to an observed association of fewer prenatal visits with a reduced likelihood of early premature delivery. In addition, women who have had medical complications or a poor pregnancy outcome in a prior pregnancy may be so anxious that they come for more prenatal visits (where problems with the fetus may be detected early), and they may also be at greater risk for a poor outcome. Thus, the potential biases can run in one or both directions. If such women are at a risk that is not amenable to prevention, an apparent association of more prenatal visits with an adverse outcome may be observed. Finally, prenatal outcome studies based on prenatal care are often biased by self-selection; that is, the women who choose to begin prenatal care early in pregnancy are often better educated and from a higher socioeconomic status with more positive attitudes toward health care. Thus, a population of women, who to begin with are at lower risk for adverse birth outcomes, select themselves for earlier prenatal care. The result is a potential for an apparent association of early prenatal care with lower risk of adverse pregnancy outcome, even if the care itself is without any true health benefit. Two Indices Used in Ecologic Studies of Health Services One index in evaluating health services that uses ecologic studies is avoidable mortality. Avoidable mortality analyses assume that the rate of “avoidable deaths” should vary inversely with the availability, accessibility, and quality of medical care in different geographic regions. The UK Office for National Statistics defines avoidable mortality as: Avoidable deaths are all those defined as preventable, amenable, or both, where each death is counted only once. Where a cause of death falls within both the preventable and amenable definition, all deaths from that cause are counted in both categories when they are presented separately. 5 Conditions include tuberculosis, hepatitis C, human immunodeficiency virus/acquired immunodeficiency syndrome (HIV/AIDS), selected malignant neoplasms, substance use disorders, cardiovascular and respiratory diseases, unintentional and intentional injuries, among others. Ideally, avoidable mortality would serve as a measure of the accessibility, adequacy, and effectiveness of care in an area. Deaths from HIV/AIDS will be less frequent in communities with ample, friendly, and convenient HIV testing and counseling and high-quality AIDS service organizations, often found in urban areas. In rural areas, such services may be less accessible, and diagnoses may only be made when a patient presents with an AIDS-defining illness. Thus, patients are more likely to have a higher mortality rate in areas with poorer service coverage, which they would not have experienced had they lived in an urban environment. Changes over time could be plotted and comparisons made with other areas. Unfortunately, the necessary data for such an analysis are often lacking for many of the conditions suggested for avoidable mortality analyses. Moreover, data on confounders may not be available and the resulting inferences may therefore be open to question. A second approach is to use health indicators. With this approach, certain sentinel conditions are assumed to reflect the general level of health care, and changes in the incidence of these conditions are plotted over time and compared with data for other populations. The changes and differences that are found are then related to changes in the health service sector