Keywords

1 Introduction

The concept of probability of detection (POD) (Sarkar et al. 1998) was initially developed to quantitatively describe the detection capabilities of nondestructive testing (NDT) systems (Blitz and Simpson 1996). A commonly used term is “90% POD” and “90% POD with 95% confidence interval”, which are written as a 90 and a 90/95 , respectively. POD curves were initially only based on experiments. The POD can be enhanced by utilizing physics-based computational models, such as the full wave ultrasonic testing simulation model (Gurrala et al. 2017), and the model-assisted probability of detection (MAPOD) methodology (Thompson et al. 2009; Aldrin et al. 2009, 2010, 2011). MAPOD can be performed using the hit/miss method (MIL-HDBK-1823), linear regression method (MIL-HDBK-1823 2009), or the Bayesian inference method (Aldrin et al. 2013; Jenson et al. 2013). Typically, the true physics-based simulation models are directly employed in the analysis.

Unfortunately, evaluating the simulation models can be time-consuming. Moreover, the MAPOD analysis process requires multiple evaluations. Consequently, the use of MAPOD with computationally expensive physics-based simulation models can be challenging to complete in a timely fashion. This has motivated the use of surrogate models (Aldrin et al. 2009, 2010, 2011; Miorelli et al. 2016; Siegler et al. 2016; Ribay et at. 2016) to alieve the computational burden. Deterministic surrogate models, such as Kriging interpolation (Aldrin et al. 2009, 2010, 2011; Du et al. 2016) and support vector regression (SVR) (Miorelli et al. 2016), have been successfully applied in this area. Stochastic surrogate models, such as polynomial chaos expansions (PCE) (Knopp et al. 2011; Sabbagh et al. 2013), are another option and have recently been utilized for MAPOD analysis (Du et al. 2017).

In this work, we integrate PCE models with least-angle regression (LAR) and hyperbolic sparse truncation schemes (Blatman et al. 2009, 2010, 2011), which can solve efficiently for the coefficients of PCE models. The proposed method is demonstrated on a spherically-void-defect NDT case, which is a benchmark case developed by the World Federal Nondestructive Evaluation Center (WFNDEC). For the purpose of this work, we use the Thompson-Gray analytical model (Gray 2012) for the ultrasonic testing simulation. The results of the MAPOD analysis using the PCE-based surrogate models are compared with direct Monte Carlo sampling (MCS) and the true model, and with MCS and deterministic Kriging surrogate models.

The paper is organized as follows. Next section gives a description of the analytical ultrasonic testing simulation model. The MAPOD analysis process is given in Sect. 3. Section 4 describes the deterministic and stochastic surrogate models. The numerical results are presented in Sect. 5. Finally, the paper ends with conclusion.

2 Ultrasonic Testing Simulation Model

The spherically-void-defect benchmark problem (shown in Fig. 1) was proposed by the WFDEC in 2004. The spherically void defect, whose radius is 0.34 mm, is included in a fused quartz block, which is surrounded by water. A spherically focused transducer, the radius of which is 6.23 mm, is used to detect this defect. The frequency range is set to be [0, 10 MHz].

Fig. 1.
figure 1

Setup of the spherically-void-defect benchmark case (left) and results of comparison between experimental data (Exp) and the analytical solution (SOV).

Full size image

The analytical model, used in this work, is known as the Thompson-Gray model (Gray 2012). This model is based on paraxial approximation of the incident and scattered ultrasonic waves, computing the spectrum of voltage at the receiving transducer in terms of the velocity diffraction coefficients of the transmitting/receiving transducers, scattering amplitude of the defect and a frequency-dependent coefficient known as the system-efficiency function (Schmerr et al. 2007). In this work, velocity diffraction coefficients were calculated using the multi-Gaussian beam model and scattering amplitude of the spherical-void was calculated using the method of separation of variables (Schmerr 2013). The system efficiency function, which is a function of the properties and settings of the transducers and the pulser, was taken from the WFNDEC archives. The time-domain pulse-echo waveforms are computed by performing FFT on the voltage spectrum. The foregoing system model was shown to be very accurate in predicting pulse-echo from the spherical void if the paraxial approximation is satisfied and radius of the void is small. To guarantee the effectiveness of this analytical model on the benchmark problem mentioned above, it is validated on this case with experimental data, given in Fig. 1, through which shows that the results match well.

3 Framework for Model-Assisted Probability of Detection

POD is essentially the quantification of inspection capability starting from the distributions of variability, and describes its accuracy with confidence bounds, also known as uncertain bounds (Spall 1997). In many cases, the final product of a POD curve is the flaw size, a, for which there is a 90% probability of detection. This flaw size is denoted a90. The 95% upper confidence bound on a90 is denoted as a90/95. The POD is typically determined through experiments which are both time-consuming and costly. This motivated the MAPOD methods with the aim for reducing the number of experimental sample points by introducing insights physics-based simulations (Thompson et al. 2009).

The main elements for generating POD curves using simulations is shown in Fig. 2. The process starts by defining the random inputs with specific statistical distributions (Fig. 2a). Next, the inputs are propagated through the simulation model (Fig. 2b). In this work, the simulation model is calculated using an analytical model (described in Sect. 2), to obtain the quantity of interest, which is the maximum signal amplitude obtained from the signal envelope (Fig. 2c). When doing detection tests for the same defect size, the results vary due to uncertainty/noise existing within the system. Usually, arbitrary number of sample runs are taken for each defect size, then a linear regression is made based on the results to obtain the so-called “\( \hat{a} \) vs. a” plot (Fig. 2d). With this information, the POD at each defect size can be obtained, thereby, the POD curves are generated (Fig. 2e).

Fig. 2.
figure 2

General process of model-assisted probability of detection: (a) probabilistic inputs; (b) simulation model; (c) response (amplitude in this work); (d) “\( \hat{a} \) vs. \( a \)” plot, (e) POD curves.

Full size image

4 Surrogate Modeling

This section describes the surrogate models used in this work. In particular, we use the deterministic Kriging interpolation surrogate model (Du et al. 2016), and the stochastic PCE surrogate models. More specifically, we use the least-angle regression (LAR) method (Blatman et al. 2010, 2011) with the hyperbolic truncation technique (Blatman et al. 2009).

4.1 Deterministic Surrogate Models via Kriging

Kriging (Ryu et al. 2002) model, also known as Gaussian process regression, is a type of interpolation method, taking all observed data as sample points and minimizing the mean square error (MSE) to reach the most appropriate model coefficients. It has the generalized formula as sum of the trend function, fT(x)β, and a Gaussian random function Z(x):

$$ y(x) = {\mathbf{f}}^{T} ({\mathbf{x}}){\varvec{\upbeta}} + Z({\mathbf{x}}),{\mathbf{x}} \in {\mathbb{R}}^{m} , $$
(1)

where f(x) = [f0(x), …, fp-1(x)]T ∈ ℝp is defined with a set of the regression basis functions, β = [β0(x), …, βp-1(x)]T ∈ ℝp denotes the vector of the corresponding coefficients, and Z(x) denotes a stationary random process with zero mean, variance and nonzero covariance. In this work, Gaussian exponential correlation function is adopted, thus the nonzero covariance is of the form

$$ Cov\left[ {Z({\mathbf{x}}),Z({\mathbf{x}}')} \right] = \sigma^{2} \exp \left[ { - \sum\limits_{k = 1}^{m} {\theta_{k} \left| {x_{k} - x_{k} '} \right|^{{p_{k} }} } } \right],\;1\, < \,p_{k} \, \le \,2 $$
(2)

where θ = [θ 1 , θ 2 , …, θ m ]T, p = [p 1 , p 2 , …, p m ]T, denote the vectors of unknown hyper model parameters to be tuned.

After further derivation (Sacks 1989), the Kriging predictor \( \hat{y}\left( {\mathbf{x}} \right) \) for any untried x can be written as

$$ \hat{y}({\mathbf{x}}) = \beta_{0} + {\mathbf{r}}^{T} ({\mathbf{x}}){\mathbf{R}}^{ - 1} ({\mathbf{y}}_{S} - \beta_{0} {\mathbf{1}}), $$
(3)

where β0 comes from generalized least squares estimation.

A unique feature of Kriging model is that it provides an uncertainty estimation (or MSE) for the prediction, which is very useful for sample-points refinement. Further details are beyond the scope of this paper, readers who have interests are suggested to go through Forrester et al. (2008).

4.2 Stochastic Surrogate Models via Polynomial Chaos Expansions

In this work, the stochastic expansions are generated using non-intrusive PCE (Xiong et al. 2010, 2011). PCE theory enables the fast construction of surrogate models, as well as an efficient statistical analysis of the model responses. More specifically, to the calculate coefficients more efficiently and accurately, we use the LAR algorithms (Blatman et al. 2010, 2011) and the hyperbolic truncation scheme (Blatman et al. 2009).

4.2.1 Generalized Polynomial Chaos Expansions

PCE is a type of stochastic surrogate model, having the generalized formulation of (Wiener 1938)

$$ Y = M({\mathbf{X}}) = \sum\limits_{i = 1}^{\infty } {\alpha_{i} {\varvec{\Psi}}_{i} ({\mathbf{X}})} , $$
(4)

where, X ∈ ℝM is a vector with random independent components, described by a probability density function f X , Y ≡ M(X) is a map of X, i is the index of ith polynomial term, Ψ is multivariate polynomial basis, and α is corresponding coefficient of basis function. In practice, the total number of sample points needed does not have to be infinite, instead, a truncated form of the PCE is used

$$ M({\mathbf{X}}) \approx M^{PC} ({\mathbf{X}}) = \sum\limits_{i = 1}^{P} {\alpha_{i} {\varvec{\Psi}}_{i} ({\mathbf{X}})} , $$
(5)

where, MPC(X) is the approximate truncated PCE model, P is the total number of required sample points and can be calculated as

$$ P = \frac{(p + n)!}{p!n!}, $$
(6)

where, p is the required order of PCE, and n is the total number of random variables.

4.2.2 Least-Angle Regression

When solving for coefficients of the PCE, this works selects state-of-the-art LAR method, which treats the observed data of actual model as a summation of PCE predictions at the same design points and corresponding residual (Efron et al. 2004)

$$ M({\mathbf{X}}) = M^{PC} ({\mathbf{X}}) + \varepsilon_{P} = \sum\limits_{i = 1}^{P} {\alpha_{i} {\varvec{\Psi}}_{i} ({\mathbf{X}})} + \varepsilon_{P} \equiv {\varvec{\upalpha}}^{T} {\varvec{\Psi}}({\mathbf{X}}) + \varepsilon_{P} , $$
(7)

where εp is the residual between M(X) and MPC(X), which is to be minimized in least-squares methods.

Then the initial problem can be converted to a least-squares minimization problem

$$ {\hat{\varvec{\alpha }}} = \arg \hbox{min} \;E[{\varvec{\upalpha}}^{T}\Psi ({\mathbf{X}}) - M({\mathbf{X}})]. $$
(8)

Adding one more regularization term to favor low-rank solution (Udell et al. 2016)

$$ {\hat{\varvec{\alpha }}} = \arg \hbox{min} \;E[{\varvec{\upalpha}}^{T} {\varvec{\uppsi}}({\mathbf{x}}) - M({\mathbf{x}})] + \lambda ||{\varvec{\upalpha}}||_{1} , $$
(9)

where λ is a penalty factor, ||α||1 is L1 norm of the coefficients of PCE. The LAR algorithm, solving for the least-squares minimization problem (Eq. (9) in this work), is very efficient in calculation, and can accept an arbitrary number of sample points.

4.2.3 Hyperbolic Truncation Technique

Commonly used basic truncation scheme has been applied to PCE as shown in Eqs. (5) and (6) to make it in a summation of finite number of terms. In order to reduce the number of sample points needed for coefficient regression, the hyperbolic truncation technique, also known as q-norm method (Blatman et al. 2009), is applied here. The main idea is to reduce the interaction terms, since they do not have much effect on the PCE prediction due to the sparsity-of-effect principle (Blatman et al. 2009).

The hyperbolic truncation technique follows the formula (Blatman et al. 2009)

$$ A^{M,p,q} = \left\{ {\alpha \in A^{M,p} :\left( {\sum\limits_{i = 1}^{M} {\alpha_{i}^{q} } } \right)^{1/q} \le p} \right\}. $$
(10)

Here, when q = 1, it is the same as basic truncation scheme, while q < 1, it can reduce the interactive terms further based on basic truncation schemes.

4.2.4 Calculation of Statistical Moments

After solving for the coefficients, statistical moments can be obtained from those coefficients directly, due to the orthonormal characteristics of PCE basis.

The mean value of PCE is (Blatman et al. 2009)

$$ \mu^{PC} = E[M^{PC} ({\mathbf{X}})] = \alpha_{1} , $$
(11)

where α1 is the coefficient of the constant basis term Ψ1 = 1. The standard deviation of PCE is

$$ \sigma^{PC} = E[(M^{PC} ({\mathbf{X}}) - \mu^{PC} )^{2} ] = \sum\limits_{i = 2}^{P} {\alpha_{i}^{2} } , $$
(12)

where it is the summation on coefficients of non-constant basis terms only.

5 Results

The proposed approach is illustrated on the spherically-void-defect benchmark problem with two uncertain parameters (see Fig. 1). In this work, the probe angle, θ, and the probe F-number, F, are considered as uncertain, with normal N(0°, 1°) and uniform U(13, 15) distributions, respectively. The distributions are shown in Fig. 3.

Fig. 3.
figure 3

Statistical distributions of uncertainty parameters: (a) F-number; (b) probe angle: θ.

Full size image

Figure 4 gives the results of the surrogate modeling construction. In particular, Fig. 4 shows the root mean square error (RMSE) as a function of the number of samples. From Fig. 4a, the LAR sparse (LARS) PCE model can reduce the RMSE value to less than 1% (also smaller than 1% σ of testing points) using 190 Latin hypercube sampling (LHS) random sample points. The Kriging interpolation model reaches the lowest RMSE value of around 10%. Figure 4b shows how the RMSE of the surrogate model varies with the defect size.

Fig. 4.
figure 4

RMSE for Kriging and LARS PCE: (a) RMSE for 0.5 mm defect; (b) RMSE for various defect sizes.

Full size image

Statistical moments are always representative of a population of samples. Figure 5 compares the convergence on the statistical moments from the PCE model, Monte Carlo sampling (MCS) with the true model, and MCS based on the Kriging model. From the figure, it can be seen that LARS PCE method has a faster convergence rate than MCS with the true model and MCS with the Kriging model with a difference in the number of sample points of around 2 orders of magnitude.

Fig. 5.
figure 5

Convergence on the statistical moments: (a) convergence on the mean; (b) convergence on the standard deviation. Here, MCSTrue model is MCS on true model, while MCSKriging is MCS on Kriging model.

Full size image

The LARS PCE models are used to generate the “\( \hat{a} \) vs. a” plot and the POD curves, as shown in Fig. 6a and b, respectively. Through the POD curves, we obtain the a 50 , a 90 , and a 90/95 information to compare the results based on the LARS PCE models with those from using MCS with the Kriging model and true model (see Table 1). We can see that the important POD metrics from the LARS PCE model match well with those from true model. More specifically, the relative differences between the LARS PCE model and the true model on a 50 , a 90 , and a 90/95 are 0.05%, 0.35%, and 0.39%, respectively. However, the relative differences between MCS with the Kriging model and MCS with the true model are −2.22%, −25.7%, −29.65%, respectively.

Fig. 6.
figure 6

POD generation using the LARS PCE model: (a) “\( \hat{a} \) vs. \( a \)” plots; (b) POD curves.

Full size image
Table 1. Comparison on the POD metrics obtained using MCS with the true model, MCS with the Kriging model, and the LARS PCE model. Here Δ is the relative difference with true model.
Full size table

6 Conclusion

In this paper, POD curves are generated through MAPOD framework. Due to the expensive time costs of physics-based simulation model, a type of stochastic surrogate model, PCE surrogate model, is integrated with LAR method and hyperbolic sparse-grid scheme. The convergence on statistical moments from PCE model is compared with actual model based Monte Carlo method, and Kriging based Monte Carlo, through which a two orders of magnitude faster convergence is obtained while Kriging based Monte Carlo is oscillating. Important metrics, namely, a 50 , a 90 , and a 90/95 , from PCE models, are also compared, and have good match with those from true model.

In future work, the surrogate-based modeling framework can be applied to more complex and time-consuming models, such as full wave model, through which the problem under test does not have to be limited as spherically void defect.