According to ecology theory, isolated habitat fragments cannot maintain populations of specialized species. Yet, empirical evidence based on monitoring of the same fragments over time is still limited. We studied the colonization–extinction dynamics of eight wood-decaying fungal species in 16 old-growth forest fragments (<14 ha) over a 20-year period (1997–2017). We observed 19 extinctions and 5 colonizations; yet, the distribution of extinctions and colonizations did not differ from the one expected by chance for any of the species. Twenty-six percent of the extinctions took place in two natural fragments amid large forest–peatland complexes. Phellinus nigrolimitatus (Romell) Bourdot and Galzin decreased and Phellinus ferrugineofuscus (P. Karst.) Bourdot increased in abundance (number of logs occupied). The volume of living spruce trees in the forest fragments correlated positively with the number of logs inhabited in five of the study species. Because fragment characteristics did not affect species turnover, it seems that stochastic processes governed colonizations and extinctions. Although the least abundant species in 1997 had declined, and the most abundant species had become more abundant, it appears that specialized wood-decaying fungi can persist for decades in isolated old-growth forest fragments, if suitable dead wood is continuously available.