One of the hallmarks of quantum statistics, tightly entwined with the concept of topological phases of matter, is the prediction of anyons. Here, the authors introduce a simple quantum impurity model, where bosonic or fermionic impurities turn into anyons as a consequence of their interaction with the surrounding bath. This model opens up simple numerical approaches to investigate the spectra of many anyons. More importantly, the model paves the way towards realizing anyons using impurities in standard condensed matter systems, such as crystal lattices and ultracold gases.