A compensatory approach to optimal selection with mastery scores

This paper presents some Bayesian theories of simultaneous optimization of decision rules for test-based decisions. Simultaneous decision making arises when an institution has to make a series of selection, placement, or mastery decisions with respect to subjects from a population. An obvious example is the use of individualized instruction in education. Compared with separate optimization, a simultaneous approach has two advantages. First, test scores used in previous decisions can be used as "prior" data in later decisions, and the efficiency of the decisions can be increased. Second, more realistic utility structures can be obtained defining utility functions for earlier decisions on later criteria. An important distinction is made between weak and strong decision rules. As opposed to strong rules, weak rules are allowed to be a function of prior test scores. Conditions for monotonicity of optimal weak and strong rules are presented. Also, it is shown that under mild conditions on the test score distributions and utility functions, weak rules are always compensatory by nature. To illustrate this approach, a common decision problem in education and psychology, consisting of a selection decision for treatment followed by a mastery decision, is analyzed