Probabilistic time series forecasting has played critical role in decision-making processes due to its capability to quantify uncertainties. Deep forecasting models, however, could be prone to input perturbations, the notion of which has not even been completely established, together with that of robustness, in the regime of probabilistic forecasting. In this work, we propose the framework of robust probabilistic time series forecasting. First, we generalize the concept of adversarial input perturbation, based on which we formulate the concept of robustness in terms of bounded Wasserstein deviation. Then we extend randomized smoothing technique to attain robust probabilistic forecasters; we provide theoretical robustness certificates on a class of adversarial additive perturbations and study its extensions. Lastly, extensive experiments demonstrate that our methods are empirically effective in enhancing the forecast quality under additive adversarial attacks and forecast consistency under supplement of noisy observations.