Search for b -> u transitions in B- -> DK- and D*K- decays

We report results from an updated study of the suppressed decays B−→DK− and B−→D∗K− followed by D→K+π−, where D(*) indicates a D(*)0 or a D⎯⎯⎯(*)0 meson, and D∗→Dπ0 or D∗→Dγ. These decays are sensitive to the Cabibbo-Kobayashi-Maskawa unitarity triangle angle γ due to interference between the b→c transition B−→D(*)0K− followed by the doubly Cabibbo-suppressed decay D0→K+π−, and the b→u transition B−→D⎯⎯⎯(*)0K− followed by the Cabibbo-favored decay D⎯⎯⎯0→K+π−. We also report an analysis of the decay B−→D(*)π− with the D decaying into the doubly Cabibbo-suppressed mode D→K+π−. Our results are based on 467×106 Υ(4S)→BB⎯⎯⎯ decays collected with the BABAR detector at SLAC. We measure the ratios ℛ(*) of the suppressed ([K+π−]DK−/π−) to favored ([K−π+]DK−/π−) branching fractions as well as the CP asymmetries (*) of those modes. We see indications of signals for the B−→DK− and B−→D∗Dπ0K− suppressed modes, with statistical significances of 2.1 and 2.2σ, respectively, and we measure: ℛDK=(1.1±0.6±0.2)×10−2,DK=−0.86±0.47+0.12−0.16, ℛ∗(Dπ0)K=(1.8±0.9±0.4)×10−2,∗(Dπ0)K=+0.77±0.35±0.12,ℛ∗(Dγ)K=(1.3±1.4±0.8)×10−2,∗(Dγ)K=+0.36±0.94+0.25−0.41, where the first uncertainty is statistical and the second is systematic. We use a frequentist approach to obtain the magnitude of the ratio rB≡|A(B−→D⎯⎯⎯0K−)/A(B−→D0K−)|=(9.5+5.1−4.1)%, with rB<16.7% at 90% confidence level. In the case of B−→D∗K− we find r∗B≡|A(B−→D⎯⎯⎯*0K−)/A(B−→D*0K−)|=(9.6+3.5−5.1)%, with r∗B