A Hybrid Fuzzy Multi-criteria Decision Making Model to Evaluate the Overall Performance of Public Emergency Departments: A Case Study

[EN] Performance evaluation is relevant for supporting managerial decisions related to the improvement of public emergency departments (EDs). As different criteria from ED context and several alternatives need to be considered, selecting a suitable Multicriteria Decision-Making (MCDM) approach has become a crucial step for ED performance evaluation. Although some methodologies have been proposed to address this challenge, a more complete approach is still lacking. This paper bridges this gap by integrating three potent MCDM methods. First, the Fuzzy Analytic Hierarchy Process (FAHP) is used to determine the criteria and sub-criteria weights under uncertainty, followed by the interdependence evaluation via fuzzy Decision-Making Trial and Evaluation Laboratory(FDEMATEL). The fuzzy logic is merged with AHP and DEMATEL to illustrate vague judgments. Finally, the Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS) is used for ranking EDs. This approach is validated in a real 3-ED cluster. The results revealed the critical role of Infrastructure (21.5%) in ED performance and the interactive nature of Patient safety (C+R =12.771). Furthermore, this paper evidences the weaknesses to be tackled for upgrading the performance of each ED.Ortiz-Barrios, M.; Alfaro Saiz, JJ. (2020). A Hybrid Fuzzy Multi-criteria Decision Making Model to Evaluate the Overall Performance of Public Emergency Departments: A Case Study. International Journal of Information Technology & Decision Making. 19(6):1485-1548. https://doi.org/10.1142/S021962202050036414851548196Lord, K., Parwani, V., Ulrich, A., Finn, E. B., Rothenberg, C., Emerson, B., … Venkatesh, A. K. (2018). Emergency department boarding and adverse hospitalization outcomes among patients admitted to a general medical service. The American Journal of Emergency Medicine, 36(7), 1246-1248. doi:10.1016/j.ajem.2018.03.043Sørup, C. M., Jacobsen, P., & Forberg, J. L. (2013). Evaluation of emergency department performance – a systematic review on recommended performance and quality-in-care measures. Scandinavian Journal of Trauma, Resuscitation and Emergency Medicine, 21(1). doi:10.1186/1757-7241-21-62Farokhi, S., & Roghanian, E. (2018). Determining quantitative targets for performance measures in the balanced scorecard method using response surface methodology. Management Decision, 56(9), 2006-2037. doi:10.1108/md-08-2017-0772Ortiz Barrios, M. A., & Felizzola Jiménez, H. (2016). Use of Six Sigma Methodology to Reduce Appointment Lead-Time in Obstetrics Outpatient Department. Journal of Medical Systems, 40(10). doi:10.1007/s10916-016-0577-3Sunder M., V., Ganesh, L. S., & Marathe, R. R. (2018). A morphological analysis of research literature on Lean Six Sigma for services. International Journal of Operations & Production Management, 38(1), 149-182. doi:10.1108/ijopm-05-2016-0273Bergeron, B. P. (2017). Performance Management in Healthcare. doi:10.4324/9781315102214Santos, S. P., Belton, V., Howick, S., & Pilkington, M. (2018). Measuring organisational performance using a mix of OR methods. Technological Forecasting and Social Change, 131, 18-30. doi:10.1016/j.techfore.2017.07.028Ho, W., & Ma, X. (2018). The state-of-the-art integrations and applications of the analytic hierarchy process. European Journal of Operational Research, 267(2), 399-414. doi:10.1016/j.ejor.2017.09.007Dargi, A., Anjomshoae, A., Galankashi, M. R., Memari, A., & Tap, M. B. M. (2014). Supplier Selection: A Fuzzy-ANP Approach. Procedia Computer Science, 31, 691-700. doi:10.1016/j.procs.2014.05.317Jing, M., Jie, Y., Shou-yi, L., & Lu, W. (2015). Application of fuzzy analytic hierarchy process in the risk assessment of dangerous small-sized reservoirs. International Journal of Machine Learning and Cybernetics, 9(1), 113-123. doi:10.1007/s13042-015-0363-4Samanlioglu, F., Taskaya, Y. E., Gulen, U. C., & Cokcan, O. (2018). A Fuzzy AHP–TOPSIS-Based Group Decision-Making Approach to IT Personnel Selection. International Journal of Fuzzy Systems, 20(5), 1576-1591. doi:10.1007/s40815-018-0474-7CHEN, M.-F., TZENG, G.-H., & TANG, T.-I. (2005). FUZZY MCDM APPROACH FOR EVALUATION OF EXPATRIATE ASSIGNMENTS. International Journal of Information Technology & Decision Making, 04(02), 277-296. doi:10.1142/s0219622005001520Gul, M., Celik, E., Gumus, A. T., & Guneri, A. F. (2016). Emergency department performance evaluation by an integrated simulation and interval type-2 fuzzy MCDM-based scenario analysis. European J. of Industrial Engineering, 10(2), 196. doi:10.1504/ejie.2016.075846Jovčić, Průša, Dobrodolac, & Švadlenka. (2019). A Proposal for a Decision-Making Tool in Third-Party Logistics (3PL) Provider Selection Based on Multi-Criteria Analysis and the Fuzzy Approach. Sustainability, 11(15), 4236. doi:10.3390/su11154236Saaty, T. L., & Vargas, L. G. (2012). Models, Methods, Concepts & Applications of the Analytic Hierarchy Process. International Series in Operations Research & Management Science. doi:10.1007/978-1-4614-3597-6Vargas, L. G. (2016). Voting with Intensity of Preferences. International Journal of Information Technology & Decision Making, 15(04), 839-859. doi:10.1142/s0219622016400058Lee, K.-C., Tsai, W.-H., Yang, C.-H., & Lin, Y.-Z. (2018). An MCDM approach for selecting green aviation fleet program management strategies under multi-resource limitations. Journal of Air Transport Management, 68, 76-85. doi:10.1016/j.jairtraman.2017.06.011Labib, A., & Read, M. (2015). A hybrid model for learning from failures: The Hurricane Katrina disaster. Expert Systems with Applications, 42(21), 7869-7881. doi:10.1016/j.eswa.2015.06.020Hosseini, S., & Khaled, A. A. (2016). A hybrid ensemble and AHP approach for resilient supplier selection. Journal of Intelligent Manufacturing, 30(1), 207-228. doi:10.1007/s10845-016-1241-yZavadskas, E. K., Govindan, K., Antucheviciene, J., & Turskis, Z. (2016). Hybrid multiple criteria decision-making methods: a review of applications for sustainability issues. Economic Research-Ekonomska Istraživanja, 29(1), 857-887. doi:10.1080/1331677x.2016.1237302Lolli, F., Balugani, E., Ishizaka, A., Gamberini, R., Butturi, M. A., Marinello, S., & Rimini, B. (2019). On the elicitation of criteria weights in PROMETHEE-based ranking methods for a mobile application. Expert Systems with Applications, 120, 217-227. doi:10.1016/j.eswa.2018.11.030De Almeida Filho, A. T., Clemente, T. R. N., Morais, D. C., & de Almeida, A. T. (2018). Preference modeling experiments with surrogate weighting procedures for the PROMETHEE method. European Journal of Operational Research, 264(2), 453-461. doi:10.1016/j.ejor.2017.08.006Sun, G., Guan, X., Yi, X., & Zhou, Z. (2018). An innovative TOPSIS approach based on hesitant fuzzy correlation coefficient and its applications. Applied Soft Computing, 68, 249-267. doi:10.1016/j.asoc.2018.04.004Frazão, T. D. C., Camilo, D. G. G., Cabral, E. L. S., & Souza, R. P. (2018). Multicriteria decision analysis (MCDA) in health care: a systematic review of the main characteristics and methodological steps. BMC Medical Informatics and Decision Making, 18(1). doi:10.1186/s12911-018-0663-1Ortiz-Barrios, M. A., Herrera-Fontalvo, Z., Rúa-Muñoz, J., Ojeda-Gutiérrez, S., De Felice, F., & Petrillo, A. (2018). An integrated approach to evaluate the risk of adverse events in hospital sector. Management Decision, 56(10), 2187-2224. doi:10.1108/md-09-2017-0917Al Salem, A. A., & Awasthi, A. (2018). Investigating rank reversal in reciprocal fuzzy preference relation based on additive consistency: Causes and solutions. Computers & Industrial Engineering, 115, 573-581. doi:10.1016/j.cie.2017.11.027Aires, R. F. de F., & Ferreira, L. (2019). A new approach to avoid rank reversal cases in the TOPSIS method. Computers & Industrial Engineering, 132, 84-97. doi:10.1016/j.cie.2019.04.023Emrouznejad, A., & Yang, G. (2018). A survey and analysis of the first 40 years of scholarly literature in DEA: 1978–2016. Socio-Economic Planning Sciences, 61, 4-8. doi:10.1016/j.seps.2017.01.008Arya, A., & Yadav, S. P. (2017). Development of FDEA Models to Measure the Performance Efficiencies of DMUs. International Journal of Fuzzy Systems, 20(1), 163-173. doi:10.1007/s40815-017-0325-yMufazzal, S., & Muzakkir, S. M. (2018). A new multi-criterion decision making (MCDM) method based on proximity indexed value for minimizing rank reversals. Computers & Industrial Engineering, 119, 427-438. doi:10.1016/j.cie.2018.03.045Kaliszewski, I., & Podkopaev, D. (2016). Simple additive weighting—A metamodel for multiple criteria decision analysis methods. Expert Systems with Applications, 54, 155-161. doi:10.1016/j.eswa.2016.01.042Mousavi-Nasab, S. H., & Sotoudeh-Anvari, A. (2018). A new multi-criteria decision making approach for sustainable material selection problem: A critical study on rank reversal problem. Journal of Cleaner Production, 182, 466-484. doi:10.1016/j.jclepro.2018.02.062Chen, Z., Ming, X., Zhang, X., Yin, D., & Sun, Z. (2019). A rough-fuzzy DEMATEL-ANP method for evaluating sustainable value requirement of product service system. Journal of Cleaner Production, 228, 485-508. doi:10.1016/j.jclepro.2019.04.145Jumaah, F. M., Zadain, A. A., Zaidan, B. B., Hamzah, A. K., & Bahbibi, R. (2018). Decision-making solution based multi-measurement design parameter for optimization of GPS receiver tracking channels in static and dynamic real-time positioning multipath environment. Measurement, 118, 83-95. doi:10.1016/j.measurement.2018.01.011Singh, A., & Prasher, A. (2017). Measuring healthcare service quality from patients’ perspective: using Fuzzy AHP application. Total Quality Management & Business Excellence, 30(3-4), 284-300. doi:10.1080/14783363.2017.1302794Otay, İ., Oztaysi, B., Cevik Onar, S., & Kahraman, C. (2017). Multi-expert performance evaluation of healthcare institutions using an integrated intuitionistic fuzzy AHP&DEA methodology. Knowledge-Based Systems, 133, 90-106. doi:10.1016/j.knosys.2017.06.028Awasthi, A., Govindan, K., & Gold, S. (2018). Multi-tier sustainable global supplier selection using a fuzzy AHP-VIKOR based approach. International Journal of Production Economics, 195, 106-117. doi:10.1016/j.ijpe.2017.10.013Gul, M., Guneri, A. F., & Nasirli, S. M. (2018). A fuzzy-based model for risk assessment of routes in oil transportation. International Journal of Environmental Science and Technology, 16(8), 4671-4686. doi:10.1007/s13762-018-2078-zKazancoglu, Y., Kazancoglu, I., & Sagnak, M. (2018). Fuzzy DEMATEL-based green supply chain management performance. Industrial Management & Data Systems, 118(2), 412-431. doi:10.1108/imds-03-2017-0121Abdullah, L., & Zulkifli, N. (2015). Integration of fuzzy AHP and interval type-2 fuzzy DEMATEL: An application to human resource management. Expert Systems with Applications, 42(9), 4397-4409. doi:10.1016/j.eswa.2015.01.021Ashtiani, M., & Azgomi, M. A. (2016). A hesitant fuzzy model of computational trust considering hesitancy, vagueness and uncertainty. Applied Soft Computing, 42, 18-37. doi:10.1016/j.asoc.2016.01.023Zyoud, S. H., & Fuchs-Hanusch, D. (2017). A bibliometric-based survey on AHP and TOPSIS techniques. Expert Systems with Applications, 78, 158-181. doi:10.1016/j.eswa.2017.02.016Scholz, S., Ngoli, B., & Flessa, S. (2015). Rapid assessment of infrastructure of primary health care facilities – a relevant instrument for health care systems management. BMC Health Services Research, 15(1). doi:10.1186/s12913-015-0838-8Ivlev, I., Vacek, J., & Kneppo, P. (2015). Multi-criteria decision analysis for supporting the selection of medical devices under uncertainty. European Journal of Operational Research, 247(1), 216-228. doi:10.1016/j.ejor.2015.05.075Kovacs, E., Strobl, R., Phillips, A., Stephan, A.-J., Müller, M., Gensichen, J., & Grill, E. (2018). Systematic Review and Meta-analysis of the Effectiveness of Implementation Strategies for Non-communicable Disease Guidelines in Primary Health Care. Journal of General Internal Medicine, 33(7), 1142-1154. doi:10.1007/s11606-018-4435-5Morley, C., Unwin, M., Peterson, G. M., Stankovich, J., & Kinsman, L. (2018). Emergency department crowding: A systematic review of causes, consequences and solutions. PLOS ONE, 13(8), e0203316. doi:10.1371/journal.pone.0203316Hermann, R. M., Long, E., & Trotta, R. L. (2019). Improving Patients’ Experiences Communicating With Nurses and Providers in the Emergency Department. Journal of Emergency Nursing, 45(5), 523-530. doi:10.1016/j.jen.2018.12.001Hawley, K. L., Mazer-Amirshahi, M., Zocchi, M. S., Fox, E. R., & Pines, J. M. (2015). Longitudinal Trends in U.S. Drug Shortages for Medications Used in Emergency Departments (2001-2014). Academic Emergency Medicine, 23(1), 63-69. doi:10.1111/acem.12838Stang, A. S., Crotts, J., Johnson, D. W., Hartling, L., & Guttmann, A. (2015). Crowding Measures Associated With the Quality of Emergency Department Care: A Systematic Review. Academic Emergency Medicine, 22(6), 643-656. doi:10.1111/acem.12682Chanamool, N., & Naenna, T. (2016). Fuzzy FMEA application to improve decision-making process in an emergency department. Applied Soft Computing, 43, 441-453. doi:10.1016/j.asoc.2016.01.007Farup, P. G. (2015). Are measurements of patient safety culture and adverse events valid and reliable? Results from a cross sectional study. BMC Health Services Research, 15(1). doi:10.1186/s12913-015-0852-xCarter, E. J., Pouch, S. M., & Larson, E. L. (2013). The Relationship Between Emergency Department Crowding and Patient Outcomes: A Systematic Review. Journal of Nursing Scholarship, 46(2), 106-115. doi:10.1111/jnu.12055Ebben, R. H. A., Siqeca, F., Madsen, U. R., Vloet, L. C. M., & van Achterberg, T. (2018). Effectiveness of implementation strategies for the improvement of guideline and protocol adherence in emergency care: a systematic review. BMJ Open, 8(11), e017572. doi:10.1136/bmjopen-2017-017572Innes, G. D., Sivilotti, M. L. A., Ovens, H., McLelland, K., Dukelow, A., Kwok, E., … Chochinov, A. (2018). Emergency overcrowding and access block: A smaller problem than we think. CJEM, 21(2), 177-185. doi:10.1017/cem.2018.446Di Somma, S., Paladino, L., Vaughan, L., Lalle, I., Magrini, L., & Magnanti, M. (2014). Overcrowding in emergency department: an international issue. Internal and Emergency Medicine, 10(2), 171-175. doi:10.1007/s11739-014-1154-8Uthman, O. A., Walker, C., Lahiri, S., Jenkinson, D., Adekanmbi, V., Robertson, W., & Clarke, A. (2018). General practitioners providing non-urgent care in emergency department: a natural experiment. BMJ Open, 8(5), e019736. doi:10.1136/bmjopen-2017-019736Razzak, J. A., Baqir, S. M., Khan, U. R., Heller, D., Bhatti, J., & Hyder, A. A. (2013). Emergency and trauma care in Pakistan: a cross-sectional study of healthcare levels. Emergency Medicine Journal, 32(3), 207-213. doi:10.1136/emermed-2013-202590Dart, R. C., Goldfrank, L. R., Erstad, B. L., Huang, D. T., Todd, K. H., Weitz, J., … Anderson, V. E. (2018). Expert Consensus Guidelines for Stocking of Antidotes in Hospitals That Provide Emergency Care. Annals of Emergency Medicine, 71(3), 314-325.e1. doi:10.1016/j.annemergmed.2017.05.021Mkoka, D. A., Goicolea, I., Kiwara, A., Mwangu, M., & Hurtig, A.-K. (2014). Availability of drugs and medical supplies for emergency obstetric care: experience of health facility managers in a rural District of Tanzania. BMC Pregnancy and Childbirth, 14(1). doi:10.1186/1471-2393-14-108Beck, M. J., Okerblom, D., Kumar, A., Bandyopadhyay, S., & Scalzi, L. V. (2016). Lean intervention improves patient discharge times, improves emergency department throughput and reduces congestion. Hospital Practice, 44(5), 252-259. doi:10.1080/21548331.2016.1254559Morais Oliveira, M., Marti, C., Ramlawi, M., Sarasin, F. P., Grosgurin, O., Poletti, P.-A., … Rutschmann, O. T. (2018). Impact of a patient-flow physician coordinator on waiting times and length of stay in an emergency department: A before-after cohort study. PLOS ONE, 13(12), e0209035. doi:10.1371/journal.pone.0209035Vermeulen, M. J., Stukel, T. A., Boozary, A. S., Guttmann, A., & Schull, M. J. (2016). The Effect of Pay for Performance in the Emergency Department on Patient Waiting Times and Quality of Care in Ontario, Canada: A Difference-in-Differences Analysis. Annals of Emergency Medicine, 67(4), 496-505.e7. doi:10.1016/j.annemergmed.2015.06.028Singh, S., Lin, Y.-L., Nattinger, A. B., Kuo, Y.-F., & Goodwin, J. S. (2015). Variation in readmission rates by emergency departments and emergency department providers caring for patients after discharge. Journal of Hospital Medicine, 10(11), 705-710. doi:10.1002/jhm.2407Källberg, A.-S., Göransson, K. E., Florin, J., Östergren, J., Brixey, J. J., & Ehrenberg, A. (2015). Contributing factors to errors in Swedish emergency departments. International Emergency Nursing, 23(2), 156-161. doi:10.1016/j.ienj.2014.10.002Riga, M., Vozikis, A., Pollalis, Y., & Souliotis, K. (2015). MERIS (Medical Error Reporting Information System) as an innovative patient safety intervention: A health policy perspective. Health Policy, 119(4), 539-548. doi:10.1016/j.healthpol.2014.12.006Norman, G. R., Monteiro, S. D., Sherbino, J., Ilgen, J. S., Schmidt, H. G., & Mamede, S. (2017). The Causes of Errors in Clinical Reasoning. Academic Medicine, 92(1), 23-30. doi:10.1097/acm.0000000000001421Lisbon, D., Allin, D., Cleek, C., Roop, L., Brimacombe, M., Downes, C., & Pingleton, S. K. (2014). Improved Knowledge, Attitudes, and Behaviors After Implementation of TeamSTEPPS Training in an Academic Emergency Department. American Journal of Medical Quality, 31(1), 86-90. doi:10.1177/1062860614545123Li, L., Georgiou, A., Vecellio, E., Eigenstetter, A., Toouli, G., Wilson, R., & Westbrook, J. I. (2015). The Effect of Laboratory Testing on Emergency Department Length of Stay: A Multihospital Longitudinal Study Applying a Cross‐classified Random‐effect Modeling Approach. Academic Emergency Medicine, 22(1), 38-46. doi:10.1111/acem.12565Telem, D. A., Yang, J., Altieri, M., Patterson, W., Peoples, B., Chen, H., … Pryor, A. D. (2016). Rates and Risk Factors for Unplanned Emergency Department Utilization and Hospital Readmission Following Bariatric Surgery. Annals of Surgery, 263(5), 956-960. doi:10.1097/sla.0000000000001536Rigobello, M. C. G., Carvalho, R. E. F. L. de, Guerreiro, J. M., Motta, A. P. G., Atila, E., & Gimenes, F. R. E. (2017). The perception of the patient safety climate by professionals of the emergency department. International Emergency Nursing, 33, 1-6. doi:10.1016/j.ienj.2017.03.003Farmer, B. (2016). Patient Safety in the Emergency Department. Emergency Medicine, 48(9), 396-404. doi:10.12788/emed.2016.0052Liu, H.-C., You, J.-X., Zhen, L., & Fan, X.-J. (2014). A novel hybrid multiple criteria decision making model for material selection with target-based criteria. Materials & Design, 60, 380-390. doi:10.1016/j.matdes.2014.03.071Kou, G., Ergu, D., & Shang, J. (2014). Enhancing data consistency in decision matrix: Adapting Hadamard model to mitigate judgment contradiction. European Journal of Operational Research, 236(1), 261-271. doi:10.1016/j.ejor.2013.11.035Keshavarz Ghorabaee, M., Amiri, M., Zavadskas, E. K., & Antucheviciene, J. (2017). Supplier evaluation and selection in fuzzy environments: a review of MADM approaches. Economic Research-Ekonomska Istraživanja, 30(1), 1073-1118. doi:10.1080/1331677x.2017.1314828Barrios, M. A. O., De Felice, F., Negrete, K. P., Romero, B. A., Arenas, A. Y., & Petrillo, A. (2016). An AHP-Topsis Integrated Model for Selecting the Most Appropriate Tomography Equipment. International Journal of Information Technology & Decision Making, 15(04), 861-885. doi:10.1142/s021962201640006xYeh, D.-Y., & Cheng, C.-H. (2016). Performance Management of Taiwan’s National Hospitals. International Journal of Information Technology & Decision Making, 15(01), 187-213. doi:10.1142/s0219622014500199Chen, T.-Y. (2014). An Interactive Signed Distance Approach for Multiple Criteria Group Decision-Making Based on Simple Additive Weighting Method with Incomplete Preference Information Defined by Interval Type-2 Fuzzy Sets. International Journal of Information Technology & Decision Making, 13(05), 979-1012. doi:10.1142/s0219622014500229Gou, X., Xu, Z., & Liao, H. (2019). Hesitant Fuzzy Linguistic Possibility Degree-Based Linear Assignment Method for Multiple Criteria Decision-Making. International Journal of Information Technology & Decision Making, 18(01), 35-63. doi:10.1142/s0219622017500377Saksrisathaporn, K., Bouras, A., Reeveerakul, N., & Charles, A. (2016). Application of a Decision Model by Using an Integration of AHP and TOPSIS Approaches within Humanitarian Operation Life Cycle. International Journal of Information Technology & Decision Making, 15(04), 887-918. doi:10.1142/s0219622015500261Hsiao, B., & Chen, L.-H. (2019). Performance Evaluation for Taiwanese Hospitals by Multi-Activity Network Data Envelopment Analysis. International Journal of Information Technology & Decision Making, 18(03), 1009-1043. doi:10.1142/s0219622018500165Saaty, T. L., & Ergu, D. (2015). When is a Decision-Making Method Trustworthy? Criteria for Evaluating Multi-Criteria Decision-Making Methods. International Journal of Information Technology & Decision Making, 14(06), 1171-1187. doi:10.1142/s021962201550025xChang, K.-H., Chang, Y.-C., & Lee, Y.-T. (2014). Integrating TOPSIS and DEMATEL Methods to Rank the Risk of Failure of FMEA. International Journal of Information Technology & Decision Making, 13(06), 1229-1257. doi:10.1142/s0219622014500758Yeh, T.-M., & Huang, Y.-L. (2014). Factors in determining wind farm location: Integrating GQM, fuzzy DEMATEL, and ANP. Renewable Energy, 66, 159-169. doi:10.1016/j.renene.2013.12.003Ortíz, M. A., Felizzola, H. A., & Isaza, S. N. (2015). A contrast between DEMATEL-ANP and