Decision framework on selecting the optimal subjective weighting method for evaluating e-learning approaches

Multi-Criteria Decision Making (MCDM) refers to making decisions in the presence of several criteria or objectives. The criteria weights particularly the subjective weights have great influence on the decisions since different weighting methods may yield different weights and ranking on the same problem. Furthermore, the Pairwise Comparisons (PCs) in Analytic Hierarchy Process (AHP) often encounter inconsistency in judgment which forces the decision maker(s) to revise the judgments. This study aims to develop a decision framework for selecting the optimal subjective weighting (SW) method to be applied in the evaluation of five e-learning approaches. Besides, this study proposes the Tripartite Relations for Overcoming Inconsistency (TROI) to address the inconsistency problem in PCs. The performances of nine SW techniques including PCs method on five identified e-learning criteria were compared. Moreover, this study also demonstrates the application of TROI method for processing inconsistency in AHP. Basically, the TROI method would use the first row of the inconsistent PC matrix to generate elements of the rest of the rows. A total of 95 participants in a selected university evaluated the importance of the criteria and rated the quality of each criterion for each of the five e-learning approaches. The optimal SW method has weights with the least total absolute differences compared to the geometric mean of all nine weights. The optimal weight was then used to select the most suitable e-learning approach by using Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS) method. The results of the study show that Graphical Weights is the optimal SW method, while Flipped Classroom is the most appropriate type of e-learning for implementation in the selected university. The proposed TROI method has helped addressing 12 inconsistent judgments in the PC matrices, while a new PC method, PC-TROI, has been established to achieve a consistent pairwise judgment. This study has successfully developed a decision framework to aid decision maker(s) in choosing the optimal SW method while proposing alternative method to AHP