All models are wrong, but some are useful. Still fewer are wrong in a way that is useful. This thesis shows that effective field theories (EFTs) are not only powerful predictive tools, but also contain the ingredients necessary to estimate their own imperfection. By developing a Bayesian model of EFT discrepancy, we provide the first statistically rigorous accounting of chiral EFT uncertainties in nuclear systems. These physics-based uncertainty estimates are shown to reduce bias in the extraction of nuclear parameters known as low-energy constants, and even to predict regimes where the EFT falls apart completely: its breakdown scale. This formalism is shown to be applicable in a wide range of circumstances; it can provide uncertainties for neutrinoless double beta decay, model the interior of neutron stars, uncover the constraints on chiral 3-body forces, extract scattering information from bound states via an improved Busch formula, and can even provide guidance on the intelligent design of experiments informed by EFTs.