AACC Improving Infusion Pump Safety Through Usability Testing Summary
AACC Improving Infusion Pump Safety Through Usability Testing Summary
AACC Improving Infusion Pump Safety Through Usability Testing Summary AACC Improving Infusion Pump Safety Through Usability Testing Summary Please see the attached reading assignment for your review. Please let me know if you have any questions or concerns. Weekly Assignments: ORDER NOW FOR CUSTOMIZED AND ORIGINAL ESSAY PAPERS Students are assigned one weekly reading assignment (ten total) to include peer-reviewed seminal or breakthrough articles specific to the upcoming lecture. Students are required to summarize the findings of the article in a single page summary (one page +/- one paragraph, single-line spacing, 11pt font) and include a comment and/or question for discussion. The comment may include how materials relate to other course concepts or examples from your own life experience. The question may address any aspect of the paper including something that was confusing or about applying the material to different contexts. The completed assignments will be submitted before the lecture when the reading will be discussed . Full credit will be given for submitting on time with reasonable content (1 point), including both summary questions and/or comment (1 point), and meeting length requirements (1 point). A brief grace period of 30 minutes or less will apply in order to account for occasional technical difficulties in submitting work. Timely submission of work is important to keep pace with the course and to allow time to review grading feedback to guide maximum performance on subsequent work. AACC Improving Infusion Pump Safety Through Usability Testing Summary improving_infusio J Nurs Care Qual Vol. 32, No. 2, pp. 141149 c 2017 Wolters Kluwer Health, Inc. All rights reserved. Copyright Improving Infusion Pump Safety Through Usability Testing Kristen E. Miller, DrPH, MSPH; Ryan Arnold, MD, MA; Muge Capan, PhD; Michele Campbell, BSN, RN, MSM, CPHQ, FABC; Susan Coffey Zern, MD, CHSE; Robert Dressler, MD, MBA; Ozioma O. Duru, PharmD, BCPS; Gwen Ebbert, MSN, BA, RN, RN-BC; Eric Jackson, Jr, MD, MBA; John Learish, CHTM, CBET, CRES; Danielle Strauss, BA; Pan Wu, PhD; Dean A. Bennett, RPh, CPHQ With the recognition that the introduction of new technology causes changes in workflow and may introduce new errors to the system, usability testing was performed to provide data on nursing practice and interaction with infusion pump technology. Usability testing provides the opportunity to detect and analyze potentially dangerous problems with the design of infusion pumps that could cause or allow avoidable errors. This work will reduce preventable harm through the optimization of health care delivery. Key words: adverse events, infusion pump, patient safety, technology, usability testing Author Affiliations: Value Institute, Christiana Care Health System, Newark, Delaware (Drs Miller, Arnold, Capan, Jackson, and Wu and Ms Strauss); Christiana Care Health System, Newark, Delaware (Drs Miller, Arnold, Capan, Zern, Dressler, Duru, Jackson, and Wu, Messrs Campbell, Learish, and Bennett, and Mss Ebbert and Strauss); Virtual Education and Simulation Training Center, Christiana Care Health System, Newark, Delaware (Dr Zern); and University of Delaware, Newark (Ms Strauss). The authors declare no conflict of interest. Supplemental digital content is available for this article. Direct URL citation appears in the printed text and is provided in the HTML and PDF versions of this article on the journals Web site (www.jncqjournal.com). Correspondence: Kristen E. Miller, DrPH, MSPH, Value Institute, Christiana Care Health System, 4755 Ogletown-Stanton Rd, Newark DE 19718 (krmiller@ christianacare.org). Accepted for publication: May 22, 2016 Published ahead of print: August 5, 2016 T HE VAST MAJORITY of hospitalized patients encounter one of the most widely used medical technologies in health care: infusion pumps. Increasingly, computercontrolled smart pumps can be programmed to deliver controlled amounts of painkillers, antibiotics, insulin, chemotherapy drugs, nutrients, or other fluids. While Christiana Care Health System had been using an advanced smart pump system-wide since 2009, the system introduced a replacement infusion pump from a different vendor in January 2015. The implementation of technologies changes the work of end users in foreseen and unforeseen ways, ultimately affecting an individuals quality of working life, such as job satisfaction and stress, and DOI: 10.1097/NCQ.0000000000000208 141 Copyright © 2017 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited. 142 JOURNAL OF NURSING CARE QUALITY/APRILJUNE 2017 perceived safety and quality of care.1-3 With the recognition that technology changes workflow and, therefore, requires changes to processes already in place, potentially introducing new errors to the system, usability testing was performed to evaluate nursing practice and interaction with infusion pump technology.AACC Improving Infusion Pump Safety Through Usability Testing Summary Infusion pumps are an indispensable tool in health care, but their complex design is recognized as a risk factor for administration errors. Because infusion pumps are frequently used to administer critical fluids, including high-risk medications, pump failures can have significant implications for patient safety.4 Adverse events may be the result of user error or related to deficiencies in device design and engineering including software defects, mechanical or electrical failures, and user-interface issues. User-interface issues are commonly considered to be the leading cause of dosing errors, frequently resulting from pump-programming errors.5-7 The scale of the problem is troubling: medication errors cause injury or death to 1% to 2% of patients admitted to the hospital in the United States annually.8 Despite numerous studies identifying user-interface issues with infusion pumps, there is an assumption regarding safe and effective use by the ultimate end user, a nurse.9 Well-known usability problems involving intravenous infusion pumps are identified as a breakdown of one or more primary steps (eg, nurses administering the wrong dose or the wrong drug) in the drug-delivery protocol.10 Automation can lead to a variety of unintended effects, such as automation surprise (when an automated system behaves in ways that the operators do not expect) and increased complacency.11 This has the paradoxical potential of actually increasing harm to patients by obviating health care providers from their perceived responsibility to check proper drug delivery. Ergonomics and human factors engineering offer useful frameworks for examining many of the mediating and moderating factors that may affect the use of equipment and out- comes of its use.12,13 Observational methodology provides rich, detailed information on tasks performed by nurses when administering medication and using various technologies. Usability testing provides direct information on the interaction between people and their work environment or tools. Simulated use can capture the complexity and preserve the context of the work environment within which health information technology is implemented. To address questions important to inform system-wide clinical implementation and evaluate issues identified by health care providers, we evaluated the new infusion pump through simulated use, an approach that provides the opportunity to detect and analyze potentially dangerous problems with the design of infusion pumps that could cause or allow avoidable errors. MATERIALS AND METHODS Testing environment To represent the intended use environment for infusion pumps, the study was conducted in the Christiana Care Health System Virtual Education and Simulation Training Center. Each room was set up like a patient room, with a manikin in each bed with an intravenous reservoir in place. Participants were instructed to use no-fluid-required infinity infusion pump tubing segments (equipment used for infusion training), removing the need to create simulated medications that had no impact on programming, the focus of the study. Each participant operated an infusion pump equipped with a central processing unit and at least 1 large volume infusion pump module. Medication labels were printed directly from the computerized physician order entry clinical system and represent label formatting in actual use. AACC Improving Infusion Pump Safety Through Usability Testing Summary The order detail was developed to represent the details a nurse would have access to via the electronic medical record. Other written materials needed to program the infusions (ie, screenshots of the computerized physician order entry order and electronic medication administration Copyright © 2017 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited. Improving Infusion Pump Safety Through Usability Testing record from the electronic medical record order) were also provided. Participant recruitment for usability testing sessions In general, an infusion pump is operated by a nurse who programs the rate and duration of medication delivery through a built-in software interface. Representative user groups include nurses from the operating room, intensive care units, and medical-surgical units. Inclusion criteria included nurses providing direct patient care having experience with this model infusion pump. Usability testing occurred in 3 rounds based on participation of nursing staff from nursing-shared decisionmaking councils. Testing took approximately 1 hour per person, requiring a substantial investment of hospital resources. Availability for participation was made possible through the nursing councils and occurred during their standing bimonthly council meetings. Over a 2-month testing period, 22 participants completed the usability testing. Task selection The first and most important criterion for selecting tasks is to use tasks that probe the potential usability problems with the infusion pump. Functional and clinical tasks were selected to represent the different types of tasks expected of health care providers. Functional tasks represent operational tasks associated with the use of the machine. Clinical tasks represent tasks necessary to achieve the medical benefit of the device. The medication safety officer and medication safety clinical pharmacist selected tasks on the basis of the potential for certain drugs to contribute to adverse outcomes in patients if the drug dosage is incorrectly calculated. The clinical task scenarios also represent true clinical scenarios that have resulted in facility adverse event reports. The different clinical scenarios represent drugs from different drug classes. Based on Christiana Care drug library creation, these drug classes have set dosing guidelines through established concentrations, dose limits, and clinical advisories. The participants role and unit 143 determine their drug library access, thereby impacting the clinical tasks conducted in the simulated use evaluation. Task assignment Tasks were assigned to participants in library profile categories on the basis of patient care unitlevel designations or specialty function. In our hospital, medications are restricted to levels on the basis of the type and degree of monitoring required: levels A, B, C, and Oncology. Level A medications need the least monitoring while level C medications (eg, vasopressors) need the highest monitoring. Oncology contains all drugs from level C with the addition of cancer-specific medications. Similarly, patient care units are assigned a level (A, B, or C) on the basis of monitoring capabilities available, nurse-to-patient ratio, and annual nursing competency requirements. Participants were asked to complete 10 functional tasks and 7 clinical tasks. Functional tasks were the same for every participant. Each clinical task was designed to address a specific drug class and drug combination. The clinical tasks associated with a subset of the libraries were randomly assigned to each participant depending on his or her access level, without repetition, such that every participant was asked to complete 7 clinical tasks to achieve a balanced outcome sample. Performance assessment Data collection included facilitator observation for data capture of objective components and subjective assessment from participants. AACC Improving Infusion Pump Safety Through Usability Testing Summary Primary data were collected at the scene and reviewed after the simulated use evaluation for secondary data collection. Performance measures (observed) included task failures and difficulties. A failure was defined as an action (or failure to act) that would lead to an undesirable treatment outcome to the patient or the user. A difficulty was defined as any instance of failure that was avoided by vigilance on the part of the user. Interface alerts and warning screens alone would not classify as a difficulty. Copyright © 2017 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited. 144 JOURNAL OF NURSING CARE QUALITY/APRILJUNE 2017 Preference assessment A subjective analysis after completion of the simulated use evaluation captured participants reaction to the interface. In addition to questions developed by the simulated use evaluation team, participants were asked questions specific to the infusion pump using the Post-Study System Usability Questionnaire (PSSUQ). The PSSUQ is a standardized instrument developed for use in scenario-based usability evaluation at IBM.14 It consists of 19 items aimed to address the following 5-system usability characteristics: quick completion of work, ease of learning, high-quality documentation and online information, functional adequacy, and rapid acquisition.15 The resulting scores are between 1 and 7, with lower scores indicating a higher degree of satisfaction. Statistical analysis Statistical analyses were performed using STATA/IC 11.0 statistical software (Stata Corp., College Station, Texas). Standard descriptive measures (ie, frequencies, means, medians, and SDs) were computed for variables in the data set. Descriptive statistics were used to describe basic features and compare different participants demographics and performance and preference assessments. Qualitative data regarding participant feedback were analyzed by the research team to identify trends in device vulnerabilities. RESULTS Participant demographics There were a total of 22 participants representing diverse units, experience, positions, and drug-leveling assignments. Before beginning the testing session, moderators documented demographic information for each participant. Usability testing was performed once the pumps had been in system-wide patient care use for more than 3 months. Participants were all previously trained on pump usability by infusion pump representatives (hands-on training). This training was required for all staff prior to use in direct patient care. The majority of participants participated in facility computer-based training (86%) and used practice pumps provided on their unit (64%). Less than half of participants had reviewed the product manual (45%), pump policy (41%), or unit policy (9%). Functional task performance During the functional task performance, 15 participants (68%) experienced at least 1 difficulty and 1 failure. Of the 10 tasks, 7 created difficulties and failures (Figure). The ability to lock and unlock the tamper resist (a button located on the back of the machine) created the most difficulty as participants noted that this is a function they were not aware of or ever used. Adjusting the audio volume also proved difficult as the function is not located under the options soft key, a common assumption. Attaching and detaching the pump models was difficult for more than 10% of participants, reflecting that this is a task not commonly completed on the unit. A commonly identified stressor was noise resulting from delays in programming. Only a few participants used the silence button while others acknowledged the beeping and ensuing anxiety. v Many participants powered down or shut off the channel to change a drug. Many noted that there is no easy way to clear a drug other than to shut down the machine. Clinical task performance There were difficulties or failures by at least 1 participant in 13 of the 15 drug scenarios (see Supplemental Digital Content, Figure, available at: http://links.lww.com/ JNCQ/A277). Drug scenarios that resulted in the most difficulties included abciximab (12/12, 100%), parenteral nutrition (13/13, 100%), intravenous immune globulin (14/14, 100%), methylprednisolone continuous infusion (not in the library) (13/13, 100%), alteplase (11/12, 92%), vasopressin (7/8, 88%), rituximab (10/12, 83%), vancomycin (8/11, 73%), fentanyl (5/7, 71%), and midazolam (5/7, 71%). Drug scenarios that resulted in the most failures included abciximab (12/12, 100%), intravenous immune globulin (13/14, Copyright © 2017 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited. Improving Infusion Pump Safety Through Usability Testing 145 Figure. Functional task performance results. 93%), methylprednisolone (10/13, 77%) (a drug not in the library), and vancomycin (8/11, 73%). Each of the 22 participants (100%) experienced at least 1 clinical difficulty and 1 clinical failure. Common causes of difficulty and failure included a lack of familiarity with the drug, difficulty locating the drug in the drug library, titration, and weightbased calculations. Overall reported ease of use and human error The majority of participants (86%, 19/22) said that they experienced usability problems during simulated use. Participants recognized safeguards including Guardrail infusion dose or rate limits, clinical advisory reminders (pop-ups), and reminders and hard stops (limits that cannot be overridden) as 82% felt that the infusion pump promotes safe and effective use by the end user (18/22). The majority of participants (64%, 14/22) felt that it would be easy to make an error with this pump considering their clinical environment. Few participants (23%, 5/22) recognized that their peers, when programming a drug they cannot find in the drug library, will use another medication or choose the basic infusion function. In doing so, there are no guardrails and no safety net. Christiana Care trained staff to avoid the use of basic infusion and use the DrugCalc feature in the pump when a medication is not in the library. Participants recognized concerns with rate and duration defaults as compared to the previous pump. Ease of use was assessed for overall use, operation, loading, and programming. The majority of participants believed that the infusion pump was very easy or easy to use (67%, 14/21), operate in general (86%, 18/21), load (86%, 18/21), and program (62%, 13/21). Three participants reported that programing the pump was difficult and 5 were neutral. Post-Study System Usability Questionnaire Our usability testing indicated that despite significant difficulty and failure in programming, participants were satisfied with the system (overall PSSUQ 2.75). Individual qualities Copyright © 2017 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited. 146 JOURNAL OF NURSING CARE QUALITY/APRILJUNE 2017 also ranked fairly high and included system usefulness (2.78), information quality (2.76), and interface quality (2.84). DISCUSSION Participant performance The simulated use evaluation proved especially useful as the majority of participants admitted experiencing problems during testing. Rather than learning about the trends in use error through adverse event reports, the team was able to proactively capture vulnerabilities in a safe environment. Through self-reflection, participants recognized issues regarding weight calculations, uncertainty when drugs are not listed in the library, a lack of a multistep function, dangers related to unfamiliar medications, and frustrations with a quick, loud, audio alarm. Overall, a lack of familiarity was the most common precursor to error or difficulty. AACC Improving Infusion Pump Safety Through Usability Testing Summary Participants expressed concern when asked to perform a clinical task (program an infusion) they rarely perform on their unit but for which their unit has the level designation, capability, and expectation to be competent to perform. The results of the simulated use evaluation showed a number of interesting vulnerabilities and potential opportunities for improvement. First, this study made clear the importance of research that systematically examines interference effects associated with switching from one smart pump to a new one with a different design approach to hardware and interface software. One common error, in particular, was the incidental substitution of rate and dose for certain medications constructed on what may be explained by muscle memory on the basis of the previous pump placement of display and programming fields for dose and rate. Christiana Care used a smart pump from a different vendor 2009 through January 2015. The dose field was at the top of the screen with the rate field displayed below. The fields are reversed in the new pump such that the rate field is at the top. Developing and integrating knowledge of spe- cific interference problems may assist in the unlearning of old interactions and learning of new tasks. Second, in nearly every difficulty and failure, participants requested assistance from colleagues including their peers, a charge nurse, nurse staff development specialist, registered nurse III (advanced education and expe Get a 10 % discount on an order above $ 100 Use the following coupon code : NURSING10
AACC Improving Infusion Pump Safety Through Usability Testing Summary
AACC Improving Infusion Pump Safety Through Usability Testing Summary AACC Improving Infusion Pump Safety Through Usability Testing Summary Please see the attached reading assignment for your review. Please let me know if you have any questions or concerns. Weekly Assignments: ORDER NOW FOR CUSTOMIZED AND ORIGINAL ESSAY PAPERS Students are assigned one weekly reading assignment (ten total) to include peer-reviewed seminal or breakthrough articles specific to the upcoming lecture. Students are required to summarize the findings of the article in a single page summary (one page +/- one paragraph, single-line spacing, 11pt font) and include a comment and/or question for discussion. The comment may include how materials relate to other course concepts or examples from your own life experience. The question may address any aspect of the paper including something that was confusing or about applying the material to different contexts. The completed assignments will be submitted before the lecture when the reading will be discussed . Full credit will be given for submitting on time with reasonable content (1 point), including both summary questions and/or comment (1 point), and meeting length requirements (1 point). A brief grace period of 30 minutes or less will apply in order to account for occasional technical difficulties in submitting work. Timely submission of work is important to keep pace with the course and to allow time to review grading feedback to guide maximum performance on subsequent work. AACC Improving Infusion Pump Safety Through Usability Testing Summary improving_infusio J Nurs Care Qual Vol. 32, No. 2, pp. 141149 c 2017 Wolters Kluwer Health, Inc. All rights reserved. Copyright Improving Infusion Pump Safety Through Usability Testing Kristen E. Miller, DrPH, MSPH; Ryan Arnold, MD, MA; Muge Capan, PhD; Michele Campbell, BSN, RN, MSM, CPHQ, FABC; Susan Coffey Zern, MD, CHSE; Robert Dressler, MD, MBA; Ozioma O. Duru, PharmD, BCPS; Gwen Ebbert, MSN, BA, RN, RN-BC; Eric Jackson, Jr, MD, MBA; John Learish, CHTM, CBET, CRES; Danielle Strauss, BA; Pan Wu, PhD; Dean A. Bennett, RPh, CPHQ With the recognition that the introduction of new technology causes changes in workflow and may introduce new errors to the system, usability testing was performed to provide data on nursing practice and interaction with infusion pump technology. Usability testing provides the opportunity to detect and analyze potentially dangerous problems with the design of infusion pumps that could cause or allow avoidable errors. This work will reduce preventable harm through the optimization of health care delivery. Key words: adverse events, infusion pump, patient safety, technology, usability testing Author Affiliations: Value Institute, Christiana Care Health System, Newark, Delaware (Drs Miller, Arnold, Capan, Jackson, and Wu and Ms Strauss); Christiana Care Health System, Newark, Delaware (Drs Miller, Arnold, Capan, Zern, Dressler, Duru, Jackson, and Wu, Messrs Campbell, Learish, and Bennett, and Mss Ebbert and Strauss); Virtual Education and Simulation Training Center, Christiana Care Health System, Newark, Delaware (Dr Zern); and University of Delaware, Newark (Ms Strauss). The authors declare no conflict of interest. Supplemental digital content is available for this article. Direct URL citation appears in the printed text and is provided in the HTML and PDF versions of this article on the journals Web site (www.jncqjournal.com). Correspondence: Kristen E. Miller, DrPH, MSPH, Value Institute, Christiana Care Health System, 4755 Ogletown-Stanton Rd, Newark DE 19718 (krmiller@ christianacare.org). Accepted for publication: May 22, 2016 Published ahead of print: August 5, 2016 T HE VAST MAJORITY of hospitalized patients encounter one of the most widely used medical technologies in health care: infusion pumps. Increasingly, computercontrolled smart pumps can be programmed to deliver controlled amounts of painkillers, antibiotics, insulin, chemotherapy drugs, nutrients, or other fluids. While Christiana Care Health System had been using an advanced smart pump system-wide since 2009, the system introduced a replacement infusion pump from a different vendor in January 2015. The implementation of technologies changes the work of end users in foreseen and unforeseen ways, ultimately affecting an individuals quality of working life, such as job satisfaction and stress, and DOI: 10.1097/NCQ.0000000000000208 141 Copyright © 2017 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited. 142 JOURNAL OF NURSING CARE QUALITY/APRILJUNE 2017 perceived safety and quality of care.1-3 With the recognition that technology changes workflow and, therefore, requires changes to processes already in place, potentially introducing new errors to the system, usability testing was performed to evaluate nursing practice and interaction with infusion pump technology.AACC Improving Infusion Pump Safety Through Usability Testing Summary Infusion pumps are an indispensable tool in health care, but their complex design is recognized as a risk factor for administration errors. Because infusion pumps are frequently used to administer critical fluids, including high-risk medications, pump failures can have significant implications for patient safety.4 Adverse events may be the result of user error or related to deficiencies in device design and engineering including software defects, mechanical or electrical failures, and user-interface issues. User-interface issues are commonly considered to be the leading cause of dosing errors, frequently resulting from pump-programming errors.5-7 The scale of the problem is troubling: medication errors cause injury or death to 1% to 2% of patients admitted to the hospital in the United States annually.8 Despite numerous studies identifying user-interface issues with infusion pumps, there is an assumption regarding safe and effective use by the ultimate end user, a nurse.9 Well-known usability problems involving intravenous infusion pumps are identified as a breakdown of one or more primary steps (eg, nurses administering the wrong dose or the wrong drug) in the drug-delivery protocol.10 Automation can lead to a variety of unintended effects, such as automation surprise (when an automated system behaves in ways that the operators do not expect) and increased complacency.11 This has the paradoxical potential of actually increasing harm to patients by obviating health care providers from their perceived responsibility to check proper drug delivery. Ergonomics and human factors engineering offer useful frameworks for examining many of the mediating and moderating factors that may affect the use of equipment and out- comes of its use.12,13 Observational methodology provides rich, detailed information on tasks performed by nurses when administering medication and using various technologies. Usability testing provides direct information on the interaction between people and their work environment or tools. Simulated use can capture the complexity and preserve the context of the work environment within which health information technology is implemented. To address questions important to inform system-wide clinical implementation and evaluate issues identified by health care providers, we evaluated the new infusion pump through simulated use, an approach that provides the opportunity to detect and analyze potentially dangerous problems with the design of infusion pumps that could cause or allow avoidable errors. MATERIALS AND METHODS Testing environment To represent the intended use environment for infusion pumps, the study was conducted in the Christiana Care Health System Virtual Education and Simulation Training Center. Each room was set up like a patient room, with a manikin in each bed with an intravenous reservoir in place. Participants were instructed to use no-fluid-required infinity infusion pump tubing segments (equipment used for infusion training), removing the need to create simulated medications that had no impact on programming, the focus of the study. Each participant operated an infusion pump equipped with a central processing unit and at least 1 large volume infusion pump module. Medication labels were printed directly from the computerized physician order entry clinical system and represent label formatting in actual use. AACC Improving Infusion Pump Safety Through Usability Testing Summary The order detail was developed to represent the details a nurse would have access to via the electronic medical record. Other written materials needed to program the infusions (ie, screenshots of the computerized physician order entry order and electronic medication administration Copyright © 2017 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited. Improving Infusion Pump Safety Through Usability Testing record from the electronic medical record order) were also provided. Participant recruitment for usability testing sessions In general, an infusion pump is operated by a nurse who programs the rate and duration of medication delivery through a built-in software interface. Representative user groups include nurses from the operating room, intensive care units, and medical-surgical units. Inclusion criteria included nurses providing direct patient care having experience with this model infusion pump. Usability testing occurred in 3 rounds based on participation of nursing staff from nursing-shared decisionmaking councils. Testing took approximately 1 hour per person, requiring a substantial investment of hospital resources. Availability for participation was made possible through the nursing councils and occurred during their standing bimonthly council meetings. Over a 2-month testing period, 22 participants completed the usability testing. Task selection The first and most important criterion for selecting tasks is to use tasks that probe the potential usability problems with the infusion pump. Functional and clinical tasks were selected to represent the different types of tasks expected of health care providers. Functional tasks represent operational tasks associated with the use of the machine. Clinical tasks represent tasks necessary to achieve the medical benefit of the device. The medication safety officer and medication safety clinical pharmacist selected tasks on the basis of the potential for certain drugs to contribute to adverse outcomes in patients if the drug dosage is incorrectly calculated. The clinical task scenarios also represent true clinical scenarios that have resulted in facility adverse event reports. The different clinical scenarios represent drugs from different drug classes. Based on Christiana Care drug library creation, these drug classes have set dosing guidelines through established concentrations, dose limits, and clinical advisories. The participants role and unit 143 determine their drug library access, thereby impacting the clinical tasks conducted in the simulated use evaluation. Task assignment Tasks were assigned to participants in library profile categories on the basis of patient care unitlevel designations or specialty function. In our hospital, medications are restricted to levels on the basis of the type and degree of monitoring required: levels A, B, C, and Oncology. Level A medications need the least monitoring while level C medications (eg, vasopressors) need the highest monitoring. Oncology contains all drugs from level C with the addition of cancer-specific medications. Similarly, patient care units are assigned a level (A, B, or C) on the basis of monitoring capabilities available, nurse-to-patient ratio, and annual nursing competency requirements. Participants were asked to complete 10 functional tasks and 7 clinical tasks. Functional tasks were the same for every participant. Each clinical task was designed to address a specific drug class and drug combination. The clinical tasks associated with a subset of the libraries were randomly assigned to each participant depending on his or her access level, without repetition, such that every participant was asked to complete 7 clinical tasks to achieve a balanced outcome sample. Performance assessment Data collection included facilitator observation for data capture of objective components and subjective assessment from participants. AACC Improving Infusion Pump Safety Through Usability Testing Summary Primary data were collected at the scene and reviewed after the simulated use evaluation for secondary data collection. Performance measures (observed) included task failures and difficulties. A failure was defined as an action (or failure to act) that would lead to an undesirable treatment outcome to the patient or the user. A difficulty was defined as any instance of failure that was avoided by vigilance on the part of the user. Interface alerts and warning screens alone would not classify as a difficulty. Copyright © 2017 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited. 144 JOURNAL OF NURSING CARE QUALITY/APRILJUNE 2017 Preference assessment A subjective analysis after completion of the simulated use evaluation captured participants reaction to the interface. In addition to questions developed by the simulated use evaluation team, participants were asked questions specific to the infusion pump using the Post-Study System Usability Questionnaire (PSSUQ). The PSSUQ is a standardized instrument developed for use in scenario-based usability evaluation at IBM.14 It consists of 19 items aimed to address the following 5-system usability characteristics: quick completion of work, ease of learning, high-quality documentation and online information, functional adequacy, and rapid acquisition.15 The resulting scores are between 1 and 7, with lower scores indicating a higher degree of satisfaction. Statistical analysis Statistical analyses were performed using STATA/IC 11.0 statistical software (Stata Corp., College Station, Texas). Standard descriptive measures (ie, frequencies, means, medians, and SDs) were computed for variables in the data set. Descriptive statistics were used to describe basic features and compare different participants demographics and performance and preference assessments. Qualitative data regarding participant feedback were analyzed by the research team to identify trends in device vulnerabilities. RESULTS Participant demographics There were a total of 22 participants representing diverse units, experience, positions, and drug-leveling assignments. Before beginning the testing session, moderators documented demographic information for each participant. Usability testing was performed once the pumps had been in system-wide patient care use for more than 3 months. Participants were all previously trained on pump usability by infusion pump representatives (hands-on training). This training was required for all staff prior to use in direct patient care. The majority of participants participated in facility computer-based training (86%) and used practice pumps provided on their unit (64%). Less than half of participants had reviewed the product manual (45%), pump policy (41%), or unit policy (9%). Functional task performance During the functional task performance, 15 participants (68%) experienced at least 1 difficulty and 1 failure. Of the 10 tasks, 7 created difficulties and failures (Figure). The ability to lock and unlock the tamper resist (a button located on the back of the machine) created the most difficulty as participants noted that this is a function they were not aware of or ever used. Adjusting the audio volume also proved difficult as the function is not located under the options soft key, a common assumption. Attaching and detaching the pump models was difficult for more than 10% of participants, reflecting that this is a task not commonly completed on the unit. A commonly identified stressor was noise resulting from delays in programming. Only a few participants used the silence button while others acknowledged the beeping and ensuing anxiety. v Many participants powered down or shut off the channel to change a drug. Many noted that there is no easy way to clear a drug other than to shut down the machine. Clinical task performance There were difficulties or failures by at least 1 participant in 13 of the 15 drug scenarios (see Supplemental Digital Content, Figure, available at: http://links.lww.com/ JNCQ/A277). Drug scenarios that resulted in the most difficulties included abciximab (12/12, 100%), parenteral nutrition (13/13, 100%), intravenous immune globulin (14/14, 100%), methylprednisolone continuous infusion (not in the library) (13/13, 100%), alteplase (11/12, 92%), vasopressin (7/8, 88%), rituximab (10/12, 83%), vancomycin (8/11, 73%), fentanyl (5/7, 71%), and midazolam (5/7, 71%). Drug scenarios that resulted in the most failures included abciximab (12/12, 100%), intravenous immune globulin (13/14, Copyright © 2017 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited. Improving Infusion Pump Safety Through Usability Testing 145 Figure. Functional task performance results. 93%), methylprednisolone (10/13, 77%) (a drug not in the library), and vancomycin (8/11, 73%). Each of the 22 participants (100%) experienced at least 1 clinical difficulty and 1 clinical failure. Common causes of difficulty and failure included a lack of familiarity with the drug, difficulty locating the drug in the drug library, titration, and weightbased calculations. Overall reported ease of use and human error The majority of participants (86%, 19/22) said that they experienced usability problems during simulated use. Participants recognized safeguards including Guardrail infusion dose or rate limits, clinical advisory reminders (pop-ups), and reminders and hard stops (limits that cannot be overridden) as 82% felt that the infusion pump promotes safe and effective use by the end user (18/22). The majority of participants (64%, 14/22) felt that it would be easy to make an error with this pump considering their clinical environment. Few participants (23%, 5/22) recognized that their peers, when programming a drug they cannot find in the drug library, will use another medication or choose the basic infusion function. In doing so, there are no guardrails and no safety net. Christiana Care trained staff to avoid the use of basic infusion and use the DrugCalc feature in the pump when a medication is not in the library. Participants recognized concerns with rate and duration defaults as compared to the previous pump. Ease of use was assessed for overall use, operation, loading, and programming. The majority of participants believed that the infusion pump was very easy or easy to use (67%, 14/21), operate in general (86%, 18/21), load (86%, 18/21), and program (62%, 13/21). Three participants reported that programing the pump was difficult and 5 were neutral. Post-Study System Usability Questionnaire Our usability testing indicated that despite significant difficulty and failure in programming, participants were satisfied with the system (overall PSSUQ 2.75). Individual qualities Copyright © 2017 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited. 146 JOURNAL OF NURSING CARE QUALITY/APRILJUNE 2017 also ranked fairly high and included system usefulness (2.78), information quality (2.76), and interface quality (2.84). DISCUSSION Participant performance The simulated use evaluation proved especially useful as the majority of participants admitted experiencing problems during testing. Rather than learning about the trends in use error through adverse event reports, the team was able to proactively capture vulnerabilities in a safe environment. Through self-reflection, participants recognized issues regarding weight calculations, uncertainty when drugs are not listed in the library, a lack of a multistep function, dangers related to unfamiliar medications, and frustrations with a quick, loud, audio alarm. Overall, a lack of familiarity was the most common precursor to error or difficulty. AACC Improving Infusion Pump Safety Through Usability Testing Summary Participants expressed concern when asked to perform a clinical task (program an infusion) they rarely perform on their unit but for which their unit has the level designation, capability, and expectation to be competent to perform. The results of the simulated use evaluation showed a number of interesting vulnerabilities and potential opportunities for improvement. First, this study made clear the importance of research that systematically examines interference effects associated with switching from one smart pump to a new one with a different design approach to hardware and interface software. One common error, in particular, was the incidental substitution of rate and dose for certain medications constructed on what may be explained by muscle memory on the basis of the previous pump placement of display and programming fields for dose and rate. Christiana Care used a smart pump from a different vendor 2009 through January 2015. The dose field was at the top of the screen with the rate field displayed below. The fields are reversed in the new pump such that the rate field is at the top. Developing and integrating knowledge of spe- cific interference problems may assist in the unlearning of old interactions and learning of new tasks. Second, in nearly every difficulty and failure, participants requested assistance from colleagues including their peers, a charge nurse, nurse staff development specialist, registered nurse III (advanced education and expe Get a 10 % discount on an order above $ 100 Use the following coupon code : NURSING10
Use Promo Code: FIRST15