Making Decisions without All the Answers

When the U.S. Environmental Protection Agency (EPA) issued a final rule addressing mercury leakage from power plants in 2011, the agency’s Administrator observed that the rule had been twenty years in the making—due in large part to continued uncertainty about the threat mercury posed to human health. The EPA’s mercury ruling illustrates a problem recently explored in an Institute of Medicine (IOM) report—how government officials should make regulatory decisions when they lack perfect answers to key questions. Responding to an EPA request for guidance, an IOM committee of experts from a variety of fields, including risk assessment, health, economics, public policy, and environmental law, wrote this report to address risk-management in uncertain contexts, with a specific emphasis on risks to human health. The IOM’s assessment shows how uncertainty can at times impede agency assessments of health risks, economic costs, and technological feasibility. This uncertainty can arise from lingering questions about varieties in statistical data, disagreements among experts about the validity of analytical models, and a fundamental lack of knowledge or disagreement about environmental, health, or other processes or assumptions underlying a risk assessment—what the report describes as “deep uncertainty.” Although experts can sometimes resolve unclear data and disagreements over models through further research, uncertainty can linger, particularly if an agency has to meet a decision within a certain specified time frame and limited budget. Faced with irresolvable uncertainty, how should agencies respond? The IOM addressed this question by describing important steps the EPA and other agencies have taken historically to address policy uncertainty and by providing suggestions for how they might improve their responses to future uncertainty. For example, in the past, the EPA has used a “default” system to resolve uncertainty in health risk assessments. Sometimes regulators are aware of what doses of a chemical lead to an effect on animals, but they may lack information about the necessary dose to cause a similar effect on humans. For the purposes of adopting regulations, the EPA uses a default rule by dividing the animals’ threshold dose by a factor of ten and then setting that level as the dosage that will have an effect on humans. The factor applied can change as regulators obtain more information. The IOM suggests a number of problems with agencies’ use of default systems, particularly when new information emerges that conflicts with the validity of the default approach adopted by an agency. Regulators may be hesitant to abandon their chosen defaults in favor of new information, for instance, by pointing out that new studies may have flaws of their own. This sort of response can cause problems when regulators’ demands for perfection stymie a new approach that is superior to the current system. The IOM report notes that, in theory, the EPA agrees with the importance of incorporating the latest scientific data into its decisions. However, in practice, according to a 2006 U.S. Government Accountability Office (GAO) report cited by the IOM, the EPA usually tends to fall back on its defaults. Overall, the IOM praises the EPA’s efforts to improve its approach to uncertainty, especially its use of new statistical tools in health risk assessment. Nonetheless, the IOM suggests that the EPA could improve this approach by empirically analyzing the accuracy of its past estimates made in the face of uncertainty, communicating with the public when making uncertain regulatory decisions, and engaging in public dialogue over the types of uncertain information that should play a role in its decisions. The IOM report repeatedly emphasizes the importance of transparently reporting an agency’s approach to uncertainty. According to the IOM, transparency can help ensure that decision-makers are conscious of the reasons behind their choices at the time they make them. Doing so may also prevent overly conservative choices and could offer the additional benefit of helping future decision-makers rapidly modify regulations to reflect new information. In particular, the report calls on the EPA to reveal discussions of uncertainty at a depth that would satisfy experts in the relevant field. The IOM committee also suggests more agency discussion of the factors, such as public sentiment, that may motivate decision-makers when data do not suggest a particular regulatory direction. For instance, the report questions how well the EPA has articulated its thinking on uncertainties stemming from “social factors,” including environmental justice and the political process. In conclusion, the IOM report recommends a number of efforts the EPA could use to improve its responses to uncertainty but notes that given the agency’s technological advances in response to uncertainty, the agency “can lead the development of uncertainty analyses in economics and technological assessment that are used for regulatory purposes.