On output independence and complementariness in rank-based multiple classifier decision systems

This study presents a theoretical analysis of output independence and complementariness between classi ers in a rank-based multiple classi er decision system in the context of the Partitioned Observation Space theory. To enable such an analysis, an Information Theoretic interpretation of a rank-based multiple classi er system is developed and basic concepts from Information Theory are applied to develop measures for output independence and complementariness. It is shown that output independence of classi ers is not a requirement for achieving complementariness between these classi ers. Namely, output independence does not imply a performance improvement by combining multiple classi ers. A condition called Dominance is shown to be important instead. The information theoretic measures proposed for output independence and complementariness are justi ed by simulated examples