Coherent Elastic Neutrino-Nucleus Scattering (CEvNS) is a Standard Model process that, although predicted for decades, has only been detected recently by the COHERENT collaboration. Now that CEvNS has been discovered, it provides a new probe for physics beyond the Standard Model. We study the potential to probe New Physics with CEvNS through the use of low temperature bolometers at a reactor source. We consider contributions to CEvNS due to a neutrino magnetic moment (NMM), Non-Standard Interactions (NSI) that may or may not change flavor, and simplified models containing a massive scalar or vector mediator. Targets consisting of Ge, Zn, Si, CaWO4, and Al2O3 are examined. We show that by reaching a percentage-level precision measurement on the CEvNS energy spectrum down to Script O(10) eV, forthcoming experiments will improve by two orders of magnitude both the CEvNS-based NMM limit and the search for new massive mediators. Additionally, we demonstrate that such dedicated low-threshold CEvNS experiments will lead to unprecedented constraints on NSI parameters (particularly when multiple targets are combined) which will have major implications for the global neutrino physics program.