Abstract
A search for an exotic decay of the Higgs boson to a pair of light pseudoscalar bosons is performed for the first time in the final state with two b quarks and two \(\tau\) leptons. The search is motivated in the context of models of physics beyond the standard model (SM), such as two Higgs doublet models extended with a complex scalar singlet (2HDM+S), which include the next-to-minimal supersymmetric SM (NMSSM). The results are based on a data set of proton-proton collisions corresponding to an integrated luminosity of 35.9 fb\(^{-1}\), accumulated by the CMS experiment at the LHC in 2016 at a center-of-mass energy of 13 TeV. Masses of the pseudoscalar boson between 15 and 60 GeV are probed, and no excess of events above the SM expectation is observed. Upper limits between 3 and 12% are set on the branching fraction \(\mathcal{B}\)(h \(\to\) aa \(\to\) 2\(\tau\)2b) assuming the SM production of the Higgs boson. Upper limits are also set on the branching fraction of the Higgs boson to two light pseudoscalar bosons in different 2HDM+S scenarios. Assuming the SM production cross section for the Higgs boson, the upper limit on this quantity is as low as 20% for a mass of the pseudoscalar of 40 GeV in the NMSSM.