This paper aims to better predict highly skewed auto insurance claims by combining candidate pred... more This paper aims to better predict highly skewed auto insurance claims by combining candidate predictions. We analyze a version of the Kangaroo Auto Insurance company data and study the effects of combining different methods using five measures of prediction accuracy. The results show the following. First, when there is an outstanding (in terms of Gini Index) prediction among the candidates, the “forecast combination puzzle” phenomenon disappears. The simple average method performs much worse than the more sophisticated model combination methods, indicating that combining different methods could help us avoid performance degradation. Second, the choice of the prediction accuracy measure is crucial in defining the best candidate prediction for “low frequency and high severity” (LFHS) data. For example, mean square error (MSE) does not distinguish well between model combination methods, as the values are close. Third, the performances of different model combination methods can differ d...
Given a dictionary of M_n initial estimates of the unknown true regression function, we aim to co... more Given a dictionary of M_n initial estimates of the unknown true regression function, we aim to construct linearly aggregated estimators that target the best performance among all the linear combinations under a sparse q-norm (0 ≤ q ≤ 1) constraint on the linear coefficients. Besides identifying the optimal rates of aggregation for these ℓ_q-aggregation problems, our multi-directional (or universal) aggregation strategies by model mixing or model selection achieve the optimal rates simultaneously over the full range of 0≤ q ≤ 1 for general M_n and upper bound t_n of the q-norm. Both random and fixed designs, with known or unknown error variance, are handled, and the ℓ_q-aggregations examined in this work cover major types of aggregation problems previously studied in the literature. Consequences on minimax-rate adaptive regression under ℓ_q-constrained true coefficients (0 ≤ q ≤ 1) are also provided. Our results show that the minimax rate of ℓ_q-aggregation (0 ≤ q ≤ 1) is basically d...
This assumption is often made to analyze parametric and nonparametric estimators, e.g., in van de... more This assumption is often made to analyze parametric and nonparametric estimators, e.g., in van der Lann, Dudoit and Keles (2004). Recall that for a positive sequence an, an estimator {f̂n}n=1 is said to converge exactly at rate {an} in probability under the L2 loss if (i) ∥∥∥f − f̂n∥∥∥ 2 = Op(an), and (ii) for every 0 < < 1, there exists c > 0 such that when n is large enough, P (∥∥∥f − f̂n∥∥∥ 2 ≥ c an ) ≥ 1− .
Given a dictionary of Mn predictors, in a random design regression setting with n observations, w... more Given a dictionary of Mn predictors, in a random design regression setting with n observations, we construct estimators that target the best performance among all the linear combinations of the predictors under a sparse lq-norm (0 ≤ q ≤ 1) constraint on the linear coefficients. Besides identifying the optimal rates of convergence, our universal aggregation strategies by model mixing achieve the optimal rates simultaneously over the full range of 0 ≤ q ≤ 1 for any Mn and without knowledge of the lq-norm of the best linear coefficients to represent the regression function. To allow model misspecification, our upper bound results are obtained in a framework of aggregation of estimates. A striking feature is that no specific relationship among the predictors is needed to achieve the upper rates of convergence (hence permitting basically arbitrary correlations between the predictors). Therefore, whatever the true regression function (assumed to be uniformly bounded), our estimators autom...
It is often reported in the forecast combination literature that a simple average of candidate fo... more It is often reported in the forecast combination literature that a simple average of candidate forecasts is more robust than sophisticated combining methods. This phenomenon is usually referred to as the “forecast combination puzzle”. Motivated by this puzzle, we explore its possible explanations, including high variance in estimating the target optimal weights (estimation error), invalid weighting formulas, and model/candidate screening before combination. We show that the existing understanding of the puzzle should be complemented by the distinction of different forecast combination scenarios known as combining for adaptation and combining for improvement. Applying combining methods without considering the underlying scenario can itself cause the puzzle. Based on our new understandings, both simulations and real data evaluations are conducted to illustrate the causes of the puzzle. We further propose a multi-level AFTER strategy that can integrate the strengths of different combin...
Model selection for quantile regression is a challenging problem. In ad-dition to the well-known ... more Model selection for quantile regression is a challenging problem. In ad-dition to the well-known general difficulty of model selection uncertainty, when quantiles at multiple probability levels are of interest, typically a single candidate does not serve all of them simultaneously. In this paper, we propose methods to combine quantile estimators. Oracle inequalities show that, at each given probabil-ity level, the combined estimators automatically perform nearly as well as the best candidate. Simulation and examples show that the proposed model combination approach often leads to a substantial gain in accuracy under global measures of performance.
This paper aims to better predict highly skewed auto insurance claims by combining candidate pred... more This paper aims to better predict highly skewed auto insurance claims by combining candidate predictions. We analyze a version of the Kangaroo Auto Insurance company data and study the effects of combining different methods using five measures of prediction accuracy. The results show the following. First, when there is an outstanding (in terms of Gini Index) prediction among the candidates, the “forecast combination puzzle” phenomenon disappears. The simple average method performs much worse than the more sophisticated model combination methods, indicating that combining different methods could help us avoid performance degradation. Second, the choice of the prediction accuracy measure is crucial in defining the best candidate prediction for “low frequency and high severity” (LFHS) data. For example, mean square error (MSE) does not distinguish well between model combination methods, as the values are close. Third, the performances of different model combination methods can differ d...
Given a dictionary of M_n initial estimates of the unknown true regression function, we aim to co... more Given a dictionary of M_n initial estimates of the unknown true regression function, we aim to construct linearly aggregated estimators that target the best performance among all the linear combinations under a sparse q-norm (0 ≤ q ≤ 1) constraint on the linear coefficients. Besides identifying the optimal rates of aggregation for these ℓ_q-aggregation problems, our multi-directional (or universal) aggregation strategies by model mixing or model selection achieve the optimal rates simultaneously over the full range of 0≤ q ≤ 1 for general M_n and upper bound t_n of the q-norm. Both random and fixed designs, with known or unknown error variance, are handled, and the ℓ_q-aggregations examined in this work cover major types of aggregation problems previously studied in the literature. Consequences on minimax-rate adaptive regression under ℓ_q-constrained true coefficients (0 ≤ q ≤ 1) are also provided. Our results show that the minimax rate of ℓ_q-aggregation (0 ≤ q ≤ 1) is basically d...
This assumption is often made to analyze parametric and nonparametric estimators, e.g., in van de... more This assumption is often made to analyze parametric and nonparametric estimators, e.g., in van der Lann, Dudoit and Keles (2004). Recall that for a positive sequence an, an estimator {f̂n}n=1 is said to converge exactly at rate {an} in probability under the L2 loss if (i) ∥∥∥f − f̂n∥∥∥ 2 = Op(an), and (ii) for every 0 < < 1, there exists c > 0 such that when n is large enough, P (∥∥∥f − f̂n∥∥∥ 2 ≥ c an ) ≥ 1− .
Given a dictionary of Mn predictors, in a random design regression setting with n observations, w... more Given a dictionary of Mn predictors, in a random design regression setting with n observations, we construct estimators that target the best performance among all the linear combinations of the predictors under a sparse lq-norm (0 ≤ q ≤ 1) constraint on the linear coefficients. Besides identifying the optimal rates of convergence, our universal aggregation strategies by model mixing achieve the optimal rates simultaneously over the full range of 0 ≤ q ≤ 1 for any Mn and without knowledge of the lq-norm of the best linear coefficients to represent the regression function. To allow model misspecification, our upper bound results are obtained in a framework of aggregation of estimates. A striking feature is that no specific relationship among the predictors is needed to achieve the upper rates of convergence (hence permitting basically arbitrary correlations between the predictors). Therefore, whatever the true regression function (assumed to be uniformly bounded), our estimators autom...
It is often reported in the forecast combination literature that a simple average of candidate fo... more It is often reported in the forecast combination literature that a simple average of candidate forecasts is more robust than sophisticated combining methods. This phenomenon is usually referred to as the “forecast combination puzzle”. Motivated by this puzzle, we explore its possible explanations, including high variance in estimating the target optimal weights (estimation error), invalid weighting formulas, and model/candidate screening before combination. We show that the existing understanding of the puzzle should be complemented by the distinction of different forecast combination scenarios known as combining for adaptation and combining for improvement. Applying combining methods without considering the underlying scenario can itself cause the puzzle. Based on our new understandings, both simulations and real data evaluations are conducted to illustrate the causes of the puzzle. We further propose a multi-level AFTER strategy that can integrate the strengths of different combin...
Model selection for quantile regression is a challenging problem. In ad-dition to the well-known ... more Model selection for quantile regression is a challenging problem. In ad-dition to the well-known general difficulty of model selection uncertainty, when quantiles at multiple probability levels are of interest, typically a single candidate does not serve all of them simultaneously. In this paper, we propose methods to combine quantile estimators. Oracle inequalities show that, at each given probabil-ity level, the combined estimators automatically perform nearly as well as the best candidate. Simulation and examples show that the proposed model combination approach often leads to a substantial gain in accuracy under global measures of performance.
Uploads