978-1-6654-7661-4/22/$31.00 ©2022 Ieee 25
978-1-6654-7661-4/22/$31.00 ©2022 Ieee 25
B. Feng, G. Pedrielli, Y. Peng, S. Shashaani, E. Song, C.G. Corlu, L.H. Lee, E.P. Chew, T. Roeder, and
P. Lendermann, eds.
Yingchieh Yeh
ABSTRACT
The ability to use sample data to generate confidence regions on quantiles is of recent interest. In particular,
developing confidence regions for multiple quantile values provides deeper information about the distribution
of underlying output data that may exhibit serial dependence. This paper presents a cancellation method
that employs overlapping batch quantile estimators to generate confidence regions. Our main theorem
characterizes the weak limit of the statistic used in constructing such confidence regions, showing in
particular that the derived weak limit deviates from the classical multivariate Student’s t and the normal
distributions depending on the number of batches and the extent of their overlap. We present limited
numerical results comparing the effect of fully overlapping versus non-overlapping batches to explore the
tradeoff between coverage probability, confidence region volume, and computational effort.
1 INTRODUCTION
Quantiles widely serve as key summary measures of random variables describing the functioning of a
system of interest, e.g., the completion time of a construction project, wait time experienced in a vehicular
traffic system, or the payouts from an insurance portfolio. Quantiles are almost always estimated using
output data generated from the system (or a simulation model of the system), making confidence bounds
on quantiles of natural interest since they quantify the “uncertainty” associated with the estimated quantile.
Due to their obvious utility, we say no more on motivating quantile confidence sets — see Dong and
Nakayama (2020) and references therein for further discussion. We emphasize that the “data” X1 , X2 , . . .
come from a time series and can exhibit serial correlation. And, while variance reduction methods can be
used as in Chu and Nakayama (2012), Nakayama (2014), Dong and Nakayama (2020) and the numerous
other references therein, we do not employ these methods here.
This paper develops what is identified in existing literature as cancellation methods to calculate
confidence regions on a steady state quantile vector. Motivated by corresponding success in the steady state
mean context, the statistic that we consider here (in the service of confidence region construction) is formed
from overlapping batches of data. Our main result characterizes the weak limit of such an overlapping batch
statistic, in particular demonstrating its dependence on the asymptotic batch size and the extent of batch
overlap, and its deviation from the classical Student’s t and normal random variables. Limited numerical
experience that we present re-affirms the advantages of using large overlapping batches.
In Section 2, we present the key problem of determining confidence regions for multiple quantiles.
Section 3 reviews related literature. Section 4 presents the main theorem and approach for deriving
confidence regions using overlapping batches, and Section 5 concludes with numerical results.
In the treatment that follows, we slightly abuse notation and use the same notation (for η) irrespective of
whether η is a scalar or a vector. This should cause no confusion since the dimension of η will be clear
from the context. Finally, we define φ -mixing (Ethier and Kurtz 2009, pp. 59) which represents the type
of dependence considered in this paper.
Definition 1 For a strictly stationary stochastic process {Xn , n ≥ 1}, denote
and n o
φn := sup |P(A | B) − P(B)| : A ∈ A1k , B ∈ Ak+n
∞
, P(B) > 0 .
26
Pasupathy, Singham, and Yeh
denote a vector of quantiles associated with F. We seek a method to construct a (1 − α)-confidence region
on ξ , that is, given α ∈ (0, 1), we seek an ellipsoid Cn ⊂ Rd constructed from the initial segment of data
{X j , 1 ≤ j ≤ n} such that P(ξ ∈ / Cn ) → α as n → ∞.
Also, observe that we have assumed {Xn , n ≥ 1} is a real-valued process. More generally, one might
assume that {Xn , n ≥ 1} is an S-valued stationary process, θk : S → R, k = 1, 2, . . . , d are functionals such
that F = (F1 , F2 , . . . , Fd ) is the distribution function associated with (θ1 (X1 ), θ2 (X1 ), . . . , θd (X1 )) ∈ Rd , and
the quantiles are
ξ j := inf{x ∈ R : Fj (x) ≥ η j }.
The confidence set construction problem is then that of finding a region
n o
Cn := y ∈ Rd : ∥An (y − Qn (η))∥ p ≤ Rn , p ≥ 1 (2)
satisfying, for fixed α ∈ (0, 1),
lim P(ξ ∈ Cn ) = 1 − α,
n→∞
where
Qn (η) ∈ Rd , Rn ∈ R, An ∈ Md+ and Qn (η), Rn , An ∈ σ (X1 , X2 , . . . , Xn ).
Different choices of An and p in (2) correspond to different shapes of the confidence region. In this paper,
we focus on the case where p = 2 and An is a symmetric positive-definite matrix, resulting in elliptical
confidence regions.
3 LITERATURE
If the sequence {X j , j ≥ 1} is φ -mixing (see Definition 1), F(ξi ) = ηi , F ′ (ξi ) > 0, 1 ≤ i ≤ d, and ∃κ > 0
and δ > 0 such that |F ′′ (x)| ≤ κ for x ∈ ∪B(ξi , δ ), then it follows, e.g., using Theorem 2, that for
η = (η1 , η2 , . . . , ηd ) ∈ (0, 1)d ,
√ −1/2 d min(ηi , η j ) − ηi η j
nΣ (Qn (η) − ξ ) → N(0, I); Σi, j = . (3)
F ′ (ξi )F ′ (ξ j )
Virtually all existing techniques for constructing confidence regions on quantiles directly or indirectly
exploit a central limit theorem (CLT) such as (3). Thus, a useful way of categorizing existing methods
for constructing quantile confidence regions is based on how the CLT in (3) is exploited, giving rise to
consistent and cancellation methods.
27
Pasupathy, Singham, and Yeh
28
Pasupathy, Singham, and Yeh
hR i
where the weighting function w : [0, 1] → R is twice differentiable and E 01 w(t)B(t) dt = 1. Calvin
and Nakayama (2013) also provide expressions for the weak limit S when the weighted area estimator is
constructed using nonoverlapping and overlapping batches. Alexopoulos et al. (2020) extend the work
of Calvin and Nakayama (2013) from i.i.d. data to dependent data relying on a geometric moment contraction
condition (GMC) (Wu 2005).
Prior to Calvin and Nakayama (2013), Alexopoulos et al. (2012) present an application of a technique
similar to cancellation (in the sense of not needing consistent estimation of the variance parameter)
toward the construction of a confidence interval on a quantile associated with the steady-state distribution
of a real-valued discrete time stochastic process. Specifically, the authors use a fixed number (b <
−1
∞) of non-overlapping batches to construct the batch quantile estimators Q j,m (η) := Fj,m (η), Fj,m (x) =
jm
m−1 ∑k=( j−1)m+1 I(−∞,x) (Xk ), j = 1, 2, . . . , b; m = n/b, and then crucially demonstrate under GMC (in lieu
of strong mixing) that as n → ∞,
!
√ d η(1 − η) ∑ℓ=∞ℓ=−∞ ρℓ
m (Q1,m (η) − ξ , Q2,m (η) − ξ , . . . , Qb,m (η) − ξ ) → N 0, Ib , (8)
f 2 (ξ )
where ρℓ = Corr(I(X1 ≤ ξ ), I(Xℓ+1 ≤ ξ )) is the lag-ℓ correlation associated with the process {I(X j ≤ ξ ), j ≥
1}. If Q̄n (η) = b−1 ∑bj=1 Q j,m (η) is the batching estimator and S̄n2 is the sample variance constructed from
√j,m , j = 1, 2, . . . , b, then under (8), the continuous mapping theorem (Billingsley 1999) assures us that
Q
b(Q̄n − ξ )/S̄n converges weakly to the Student’s t random √ variable with b − 1 degrees of freedom,
yielding the (1 − α) confidence interval Q̄n ± tα/2,b−1 S̄n / b. Sequest (Alexopoulos et al. 2019) and
Sequem (Alexopoulos et al. 2017) incorporate notable implementation enhancements to the essential idea
introduced in Alexopoulos et al. (2012).
The idea presented in Alexopoulos et al. (2012) is of particular relevance to what we present here. In fact,
the main theorem that we present can be seen as replacing the batching estimator Q̄n used in Alexopoulos
et al. (2012) with the sectioning estimator Qn , and then generalizing along the following three directions:
(i) allowing any degree of batch overlap ranging from fully overlapping to non-overlapping; (ii) ξ ∈ Rd
implying that the confidence regions reside in an arbitrarily high but finite dimension; and (iii) number
of batches b can be finite or infinite depending on the extent of batch overlap. As our main theorem will
make clear, (i), (ii), and (iii) cause deviations from the classical Student’s t weak limit, and thereby the
nature of the constructed confidence region.
29
Pasupathy, Singham, and Yeh
While Theorem 2 is our essential instrument to handle dependence, we could have equally used a
number of other Bahadur representations (Wu 2005) that have appeared recently.
4 MAIN THEOREM
An estimator Σn of the d × d asymptotic covariance matrix Σ appearing in (3) can be constructed using
overlapping batches of data as follows. Suppose we partition the data into possibly overlapping batches of
size mn and having initial observations offset by an amount dn . (See Figure 1 for a clear idea.) The i-th
batch consists of data X j , j = (i − 1)dn + 1, (i − 1)dn + 2, . . . , (i − 1)dn + mn where i = 1, 2, . . . , bn and the
number of batches bn = dn−1 (n − mn ) + 1. The empirical distribution from the ith batch is then
dn + 1 dn + mn
batch 2
batch 1 batch 3
1 mn 2dn + 1
Figure 1: The figure depicts partially overlapping batches. Batch 1 consists of observations X j , j =
1, 2, . . . , mn ; batch 2 consists of observations X j , j = dn + 1, dn + 2, . . . , dn + mn , and so on, with batch
i consisting X j , j = (i − 1)dn + 1, (i − 1)dn + 2, . . . , (i − 1)dn + mn . There are thus bn := dn−1 (n − mn ) + 1
batches in total, where n is the size of the dataset.
1 (i−1)dn +mn
Fi,mn (x) = ∑ I(−∞,x] (Xk ),
mn k=(i−1)d +1n
The factor (1 − mn /n)−1 ensures that the estimator Σmn is asymptotically unbiased. Also define the
corresponding “Studentized random vector”
√ − 21
Tmn := n Σmn (Qn (η) − ξ ).
We are now ready to state the main result that characterizes the weak convergence behavior of the
matrix sequence {Σmn , n ≥ 1} and the vector sequence {Tmn , n ≥ 1}.
Theorem 3 (OB Limits χOB 2 (β , b) and T (β , b)) Suppose that the postulates of Theorem 2 hold and that
OB
mn
β := lim > 0; b := lim bn ∈ {2, 3, . . . , ∞}.
n n n
30
Pasupathy, Singham, and Yeh
where {Wd (t), 0 ≤ t ≤ 1} is the d-dimensional standard Wiener process. Then the sequences {Σmn , n ≥
1}, {Tmn , n ≥ 1} satisfy
d √ √
2 (β , b) ΣT ; d−1
Σmn → Σ χOB Tmn → χOB (β , b)Wd (1) =: TOB (β , b), (11)
where
(
1 1 R 1−β
κ1 (β ,b) β (1−β ) 0 B(u, β )B(u, β )T du b=∞
χ2 OB (β , b) := 1 11 b T
κ1 (β ,b) β b ∑ j=1 B(c j , β )B(c j , β ) b ∈ N\{1},
−1
1
2 (β , b) − 2 , and where κ (β , b) = (1 − β ).
c j = ( j − 1)(1 − β )/(b − 1), and χOB (β , b) := χOB 1
Proof Sketch. Recall Equation (10) and substitute β = mn /n. Since Σ is a variance matrix there exists
√ √ √ T
Σ such that Σ = Σ Σ and define
−1 n − mn n − mn
B̃ j,mn := mn Wd ( j − 1) + mn −Wd ( j − 1) , j = 1, 2, . . . , bn .
bn − 1 bn − 1
Then,
mn bn h √ √ √ √ i
(1 − β )Σmn = ∑ (Q j,mn (η) − Qn )(Q j,mn (η) − Qn )T − ( ΣB̃ j,mn − Σn−1Wd (n))( ΣB̃ j,mn − Σn−1Wd (n))T
bn j=1
√ bn √ √ T !
√ T
1 √ mn √ mn
+ Σ× ∑ mn B̃ j,mn − n Wd (n) mn B̃ j,mn − Wd (n) × Σ =: En + Ln .
bn j=1 n
wp1
Using Theorem 2, we can show after lots of algebra that En → 0, and that Ln converges weakly to the
appropriate limit as bn → b = ∞. A similar calculation holds for the b < ∞ context.
(a) Notice that χOB2 (β , b) ∈ M + is a random matrix and T (β , b) := χ −1 (β , b)W (1) ∈ Rd is a random
d OB OB
vector. They should be seen as the χ 2 and Student’s t analogues for the context of constructing
confidence regions on objects other than the population mean.
(b) The matrix Σmn does not consistently estimate the covariance matrix Σ. This is due to the increased
variance stemming from the use of large batches as connoted by β > 0. It is in this sense that we
can “get away with” using large batches and not estimating the covariance matrix consistently. It
can be shown that if β = 0, that is, if mn /n → 0, then χOB 2 (β , b) degenerates to the identity matrix
31
Pasupathy, Singham, and Yeh
The section estimator Qn in Theorem 3 can be replaced by what has been called the batching estima-
tor (Nakayama 2014):
1 bn
Q̄n := ∑ Q j,mn (η).
bn j=1
The batching estimator Q̄n has a higher bias than the sectioning estimator Qn , and recent analysis by He
and Lam (2021) suggests sectioning estimators may have better coverage when there are a large number
of batches. The batching estimator and the batch quantiles in (9) together suggest the following alternative
to Σmn when estimating Σ:
bn
1 mn
Σ̄mn :=
κ2 (β , b) bn ∑ (Q j,m (η) − Q̄n )(Q j,m (η) − Q̄n )T ,
n n
j=1
where κ2 (β , b) is the “bias correction” factor. A theorem analogous to Theorem 3 but with Q̄n replacing
Qn (η), and with Σ̄mn replacing Σmn , yields parallel results which will be discussed in a forthcoming paper.
5 EXPERIMENTAL RESULTS
This section presents the results of numerical experiments to explore the effects of increasing batch sizes
and the effect of overlapping batches. For sample size n, the batch size is mn = β n. For nonoverlapping
batches, the number of batches is bn = n/mn and the limiting number of batches as n → ∞ is b = β −1 . For
overlapping batches, we take the maximum level of overlap, so set dn = 1 and set bn = n(1 − β ) + 1. In this
case, the limiting number of batches as n → ∞ is b = ∞. We report the coverage of confidence intervals when
estimating a single quantile, as well as the size of the confidence interval regions when d ≥ 1. Note that
we use our same estimator for nonoverlapping batches as for overlapping batches (varying the parameter
dn to adjust the level of overlap). Future work will compare the effects of numerous other nonoverlapping
batch estimators in the literature that employ alternative cancellation or consistent estimation methods.
32
Pasupathy, Singham, and Yeh
Table 1 shows that increasing the value of β yields greater coverage and greater halfwidths. Meanwhile,
the OLB approach has slightly worse coverage, but narrower half-widths. Using the same data length n,
overlapping batches with a larger β may deliver adequate coverage with smaller half-widths compared
to using NOLB methods. We observe that for small β = 0.01, the performance of nonoverlapping and
overlapping are similar, and the discrepancy increases with larger batches as the effect of the overlap
becomes greater.
Next, we explore simultaneous confidence intervals for multiple quantiles of i.i.d. data where a confidence
region is generated. Table 2 displays these results for the exponential distribution (with rate 1) for
simultaneous confidence intervals for pi = 0.01, 0.30, 0.50, 0.70, 0.99, with dimension d = 5. We use the
estimator Σmn as the estimate of the covariance matrix Σ and report the mean observed root half-volume
of the multidimensional confidence ellipsoid. Table 2 reveals that using overlapping batches appears to
reduce the volume of the confidence regions. Values marked N/A imply the batch size was not large enough
to generate a d-dimensional estimate. Larger confidence interval volumes for larger β may be related
to an observation in Corollary 4.16 of Glynn and Iglehart (1990), in which consistent estimators achieve
lower expected half-widths for the steady state mean than cancellation estimators such as standardized time
series.. We note some overcoverage for large batch sizes (large β when n is large and d > 1) when the
sample size n is not large enough, for example, in the NOLB results coverage is high when n = 1, 000 but
decreases towards nominal as n increases. The OLB results take much longer to compute so we cannot
see the convergence to nominal given computing limits.
33
Pasupathy, Singham, and Yeh
Table 2: Independent data, d = 5: Coverage values for 95% confidence intervals with mean root half-volumes
in parentheses for the joint quantile estimate for pi = 0.01, 0.30, 0.50, 0.70, 0.99 of i.i.d. exponential data
using 2000 independent replications.
NOLB
n β = 0.01 β = 0.05 β = 0.10
100 N/A 0.799 (0.287) 0.915 (0.383)
200 N/A 0.876 (0.189) 0.951 (0.272)
1,000 0.795 (0.073) 0.956 (0.082) 0.967 (0.112)
10,000 0.954 (0.022) 0.956 (0.025) 0.961 (0.035)
100,000 0.950 (0.007) 0.951 (0.008) 0.957 (0.011)
1,000,000 0.949 (0.002) 0.947 (0.002) 0.952 (0.003)
OLB
n β = 0.01 β = 0.05 β = 0.10
100 N/A 0.814 (0.275) 0.897 (0.286)
200 N/A 0.861 (0.178) 0.924 (0.199)
1,000 0.828 (0.072) 0.960 (0.077) 0.968 (0.082)
10,000 0.952 (0.021) 0.956 (0.023) 0.966 (0.025)
Table 3: Dependent data, d = 1: Coverage values for 95% confidence intervals with mean half-widths in
parentheses for the joint quantile estimate for p = 0.90 with AR(1) data using 2000 independent replications.
NOLB
n β = 0.01 β = 0.05 β = 0.10 β = 0.20
100 0.830 (0.370) 0.893 (0.458) 0.918 (0.519) 0.937 (0.663)
200 0.875 (0.289) 0.899 (0.342) 0.930 (0.395) 0.938 (0.488)
1,000 0.905 (0.145) 0.952 (0.173) 0.943 (0.189) 0.959 (0.225)
OLB
n β = 0.01 β = 0.05 β = 0.10 β = 0.20
100 0.820 (0.369) 0.898 (0.443) 0.920 (0.493) 0.939 (0.573)
200 0.877 (0.291) 0.898 (0.331) 0.929 (0.375) 0.935 (0.428)
1,000 0.919 (0.146) 0.941 (0.168) 0.953 (0.180) 0.955 (0.194)
Table 4 shows results for d = 3 where the goal is to estimate simultaneously the 0.90, 0.95, and 0.99
quantiles for AR(1) dependent data. We observe higher sample sizes are needed for higher dimensional
settings to achieve adequate coverage. While the overlapping approach with large batch sizes appear to
achieve better coverage with smaller sample sizes, we note the computation time is much higher than for
nonoverlapping interval estimation.
ACKNOWLEDGEMENTS
The authors are grateful to the anonymous referees of this paper, who suggested many comments that greatly
improved the content and presentation of this work. The first author thanks the Department of Computer
Science and Engineering, IIT Madras, Chennai, INDIA and the Operations Research Department, Naval
Postgraduate School, Monterey, CA for the hospitality during his sabbatical visit in 2021–2022.
34
Pasupathy, Singham, and Yeh
Table 4: Dependent data, d = 3: Coverage values for 95% confidence intervals with mean half-volumes in
parentheses for the joint quantile estimate for p = 0.90, 0.95, 0.99 with AR(1) data using 2000 independent
replications.
NOLB
n β = 0.01 β = 0.05 β = 0.10
500 N/A 0.834 (0.242) 0.948 (0.330)
1,000 0.501 (0.114) 0.927 (0.195) 0.946 (0.238)
2,000 0.664 (0.095) 0.928 (0.141) 0.956 (0.175)
4,000 0.832 (0.081) 0.943 (0.104) 0.958 (0.126)
OLB
n β = 0.01 β = 0.05 β = 0.10
500 N/A 0.812 (0.232) 0.940 (0.288)
1,000 0.470 (0.114) 0.925 (0.187) 0.949 (0.207)
2,000 0.672 (0.095) 0.932 (0.135) 0.947 (0.152)
4,000 0.851 (0.080) 0.946 (0.100) 0.965 (0.110)
REFERENCES
Alexopoulos, C., N. T. Argon, D. Goldsman, G. Tokol, and J. R. Wilson. 2007. “Overlapping Variance Estimators for Simulation”.
Operations Research 55(6):1090–1103.
Alexopoulos, C., J. H. Boone, D. Goldsman, A. Lolos, K. D. Dingeç, and J. R. Wilson. 2020. “Steady-State Quantile Estimation
Using Standardized Time Series”. In Proceedings of the 2020 Winter Simulation Conference, edited by K.-H. Bae, B. Feng,
S. Kim, S. Lazarova-Molnar, Z. Zheng, T. Roeder, and R. Theising, 289–300. Piscataway, New Jersey: Institute of
Electrical and Electronics Engineers, Inc.
Alexopoulos, C., D. Goldsman, A. C. Mokashi, K.-W. Tien, and J. R. Wilson. 2019. “Sequest: A Sequential Procedure for
Estimating Quantiles in Steady-State Simulations”. Operations Research 67(4):1162–1183.
Alexopoulos, C., D. Goldsman, A. C. Mokashi, and J. R. Wilson. 2017. “Automated Estimation of Extreme Steady-State Quantiles
via the Maximum Transformation”. ACM Transactions on Modeling and Computer Simulation (TOMACS) 27(4):1–29.
Alexopoulos, C., D. Goldsman, and J. R. Wilson. 2012. “A New Perspective on Batched Quantile Estimation”. In Proceedings
of the 2012 Winter Simulation Conference, edited by C. Laroque, J. Himmelspach, R. Pasupathy, O. Rose, and A. M.
Uhrmacher, 1–11. Piscataway, New Jersey: Institute of Electrical and Electronics Engineers, Inc.
Bahadur, R. R. 1966. “A Note on Quantiles in Large Samples”. The Annals of Mathematical Statistics 37(3):577–580.
Billingsley, P. 1999. Convergence of Probability Measures. 2nd ed. ed. Wiley series in Probability and Statistics. New York:
Wiley.
Calvin, J. M., and M. K. Nakayama. 2013. “Confidence Intervals for Quantiles with Standardized Time Series”. In Proceedings
of the 2013 Winter Simulations Conference, edited by R. Pasupathy, S.-H. Kim, A. Tolk, R. Hill, and M. E. Kuhl.,
601–612. Piscataway, New Jersey: Institute of Electrical and Electronics Engineers, Inc.
Chu, F., and M. K. Nakayama. 2012. “Confidence Intervals for Quantiles when Applying Variance-Reduction Techniques”.
ACM Transactions on Modeling and Computer Simulation (TOMACS) 22(2):1–25.
Dong, H., and M. K. Nakayama. 2020. “A Tutorial on Quantile Estimation via Monte Carlo”. In Monte Carlo and Quasi-Monte
Carlo Methods, Springer Proceedings in Mathematics & Statistics, 3–30. Cham: Springer International Publishing.
Ethier, S. N., and T. G. Kurtz. 2009. Markov Processes: Characterization and Convergence, Volume 282. John Wiley & Sons.
Glynn, P., and D. Iglehart. 1985. “Large-Sample Theory for Standardized Time Series: An Overview”. In Proceedings of the
1985 Winter Simulation Conference, edited by D. Gantz, G. Blais, and S. Solomon, 129–134: ACM.
Glynn, P. W. 1996. “Importance Sampling for Monte Carlo Estimation of Quantiles”. In Mathematical Methods in Stochastic
Simulation and Experimental Design: Proceedings of the 2nd St. Petersburg Workshop on Simulation, 180–185. St.
Petersberg, Russia: Publishing House of St. Petersberg University.
Glynn, P. W., and D. L. Iglehart. 1990. “Simulation Output Analysis Using Standardized Time Series”. Mathematics of
Operations Research 15(1):1–16.
Goldsman, D., M. Meketon, and L. Schruben. 1990. “Properties of Standardized Time Series Weighted Area Variance Estimators”.
Management Science 36(5):602–612.
35
Pasupathy, Singham, and Yeh
He, S., and H. Lam. 2021. “Higher-Order Coverage Error Analysis for Batching and Sectioning”. In Proceedings of the 2021
Winter Simulation Conference, edited by S. Kim, B. Feng, K. Smith, S. Masoud, Z. Zheng, C. Szabo, and M. Loper,
1–12. Piscataway, New Jersey: Institute of Electrical and Electronics Engineers, Inc.
Lei, L., C. Alexopoulos, Y. Peng, and J. R. Wilson. 2020. “Confidence Intervals and Regions for Quantiles using Conditional
Monte Carlo and Generalized Likelihood Ratios”. In Proceedings of the 2020 Winter Simulation Conference, edited by
K.-H. Bae, B. Feng, S. Kim, S. Lazarova-Molnar, Z. Zheng, T. Roeder, and R. Theising, 2071–2082. Piscataway, New
Jersey: IEEE: Institute of Electrical and Electronics Engineers, Inc.
Lei, L., C. Alexopoulos, Y. Peng, and J. R. Wilson. 2022. “Estimating Confidence Intervals and Regions for Quantiles by
Monte Carlo Simulation”. Preprint available at SSRN.
Muñoz, D. F. 2010. “On the Validity of the Batch Quantile Method for Markov Chains”. Operations Research Letters 38(3):223–
226.
Nakayama, M. K. 2014. “Confidence Intervals for Quantiles using Sectioning when Applying Variance-Reduction Techniques”.
ACM Transactions on Modeling and Computer Simulation (TOMACS) 24(4):1–21.
Schruben, L. W. 1983. “Confidence Interval Estimation Using Standardized Time Series”. Operations Research 31(6):1090–1108.
Sen, P. K. 1972. “On the Bahadur Representation of Sample Quantiles for Sequences of ϕ-Mixing Random Variables”. Journal
of Multivariate Analysis 2(1):77–95.
Serfling, R. J. 2009. Approximation Theorems of Mathematical Statistics, Volume 162. John Wiley & Sons.
Wu, W. B. 2005. “On the Bahadur Representation of Sample Quantiles for Dependent Sequences”. The Annals of Statis-
tics 33(4):1934–1963.
AUTHOR BIOGRAPHIES
RAGHU PASUPATHY is Professor of Statistics at Purdue University. His current research interests lie broadly in general
simulation methodology, statistical computation (especially optimization) and statistical inference. He has been actively involved
with the Winter Simulation Conference for the past 15 years. Raghu Pasupathy’s email address is pasupath@purdue.edu, and
his web page https://web.ics.purdue.edu/∼pasupath contains links to papers, software codes, and other material.
DASHI I. SINGHAM is a Research Associate Professor in the Operations Research Department at the Naval Postgraduate School.
She obtained her Ph.D. in Industrial Engineering & Operations Research from the University of California at Berkeley and her
B.S.E. in Operations Research and Financial Engineering from Princeton University. Her email address is dsingham@nps.edu,
and her website is https://faculty.nps.edu/dsingham/.
YINGCHIEH YEH is an Associate Professor in Institute of Industrial Management at National Central University, Taiwan.
He received his Ph.D. from the School of Industrial Engineering at Purdue University. His primary research interests
include simulation output analysis, applied probability and statistics, and applied operations research. His email address is
yeh@mgt.ncu.edu.tw.
36