Assessment of the Reliability of Reserves Estimates of Public Companies in the U.S. and Canada.

Estimation of reserves is a process used to quantify the volumes of hydrocarbon fluids that can be recovered economically from a reservoir, field, area or region, from a given date forward. A considerable level of uncertainty is involved throughout the reserves-estimation process. Unfortunately, individuals are poor at assessing uncertainty, with a common tendency for overconfidence (underestimation of uncertainty) and optimism. There are a few studies that address the reliability of reserves estimates, but none of them quantify the reliability of these estimates. This research aims to assess quantitatively the reliability of reserves estimates of public companies filing in the U.S. and Canada. To do this I measured biases in reported reserves estimates for 34 companies filing in Canada and 32 companies filing in the U.S. over the time period 2007 to 2017. Canadian companies explicitly report technical revisions of proved (1P) and proved-plus-probable (2P) reserves. U.S. companies do not report “technical revisions,” but instead report “revisions of previous estimates” and revisions due to price changes of proved (1P) reserves separately. I calculated Revisions Other Than Price (ROTP) by subtraction for U.S. companies and assumed the difference was the same as “technical revisions.” Based on probabilistic reserves definitions, it is reasonable to assume that proved reserves estimates are expected to have positive technical revisions 90% of the time, while proved- plus-probable reserves estimates are expected to have positive revisions 50% of the time. The reliability of proved and proved-plus-probable reserves estimates was assessed using calibration plots, in which the frequency of positive technical revisions is plotted against the estimate probability. Calibration plots can be used to measure confidence bias, ranging from underconfidence to complete overconfidence, and directional bias, ranging from complete pessimism to complete optimism. “Technical revisions” reported by 34 Canadian companies for the 11-year period were positive an average of 72% for 1P reserves and an average of 54% for 2P reserves, whereas the expected values were 90% and 50%, respectively. Thus, on average over this time period, filers in Canada overestimated 1P reserves and underestimated 2P reserves. Considering the entire reserves distributions, bias measurements indicate that filers in Canada were moderately overconfident and slightly pessimistic. Revisions Other Than Price (ROTP) calculated for 32 U.S. companies for the 11-year period were positive an average of only 51% for 1P reserves, compared to an expected 90%. Thus, on average over this time period, filers in the U.S. overestimated 1P reserves significantly. Considering the entire reserves distributions, bias measurements indicate that filers in the U.S. were somewhere between complete overconfidence and neutral directional bias, and moderate overconfidence and complete optimism. The biases in reserves estimates filed in both Canada and the U.S. suggest that adjustments in reserves estimation procedures are warranted. Three groups of professionals can benefit from this study: (1) estimators, who can use the methodology to track their technical revisions over time, calibrate them, and use this information to adjust future estimation procedures; (2) investors, who can analyze reported reserves estimates to compare volumes fairly; and (3) regulators, who can ensure that filers are complying with appropriate criteria for 1P and 2P reserves