Statistical treatment of test data:

Geostatistical methods
ƒ Sources of uncertainty and variability of soil
properties
ƒ Review of probability concepts
ƒ Characterizing spatial variability of soil properties
ƒ Geostatistical analysis of CPT data
ƒ Site investigation strategy
ƒ Appendix I: Geostatistics (Fenton 1997)
ƒ Appendix II: Probabilistic methods in geotechnical
engineering (US National Research Council 1995)

CIVL576/Zhang 1
 References
• Lunne, T., Robertson, P.K., and Powell J.J.M. 1997. Cone
penetration testing in geotechnical practice. Spon Press,
London.
• Fenton, Gordon A. 1997. Data analysis/Geostatistics.
Workshop presented at ASCE GeoLogan’97 Conference,
Logan, Utah, July 15, 1997.
• National Research Council. 1995. Probabilistic methods in
geotechnical engineering. The US National Research
Council, Washington DC.

CIVL576/Zhang 2
 Sources of uncertainty
Two types of uncertainties affect a soil property within a geologic layer:
• Aleatory (or inherent) uncertainty, which represents the natural
randomness of a property. For example, the variation in the ocean
wave height and the variation in a soil property in the horizontal
direction are aleatory uncertainties. This type of uncertainty cannot
be reduced.
• Epistemic uncertainty, which represents the uncertainty due to
lack of knowledge for a given property. Measurement uncertainty and
model uncertainty are epistemic uncertainties. This type of
uncertainty can be reduced, by for example increasing the number of
tests, improving the measurement method or evaluating a calculation
procedure with model tests.

CIVL576/Zhang 3
 • The uncertainties associated with soil characteristics are
generally attributed to two primary sources:
ƒ Inherent (natural soil) variability, and
ƒ Sampling and testing errors (identified as noises)
• The soil variability component and noise are separated
with the use of an autocorrelation function.

CIVL576/Zhang 4
 Statistical treatment of test data:
Geostatistical methods
ƒ Sources of uncertainty and variability of soil
properties
ƒ Review of probability concepts
ƒ Characterizing spatial variability of soil properties
ƒ Geostatistical analysis of CPT data
ƒ Site investigation strategy
ƒ Appendix I: Geostatistics (Fenton 1997)
ƒ Appendix II: Probabilistic methods in geotechnical
engineering (US National Research Council 1995)

CIVL576/Zhang 5
 Probability concepts: Mean, variance,
histogram and probability density function
Conventional statistics include
developing a histogram and
obtaining mean and coefficient of
variation. An idealized example for
cone penetration resistance is
shown in Figure 1. When enough
data are available, the probability
distribution function should be
established. In geotechnics, the
normal and lognormal distributions
are most commonly used. CCone penetration
6 7resistance
8 (MPa)9

Figure 1. Histogram of cone

penetration resistance

CIVL576/Zhang 6
 Probability concepts: 3-σ rule
• For a normal variable, the
chances that variable is
between the mean plus/minus
2σ and 3σ are 95.4% and
99.7%, respectively.
• If the most conceivable upper
bound (Lu) and lower bound
(Ll) can be estimated, then σ
can be estimated as
σ = (Lu- Ll)/6
• In most cases, the σ value
should be smaller.

FFigure 2
CIVL576/Zhang 7
 Probability concepts: Short-cut estimate
• The mean is obtained by averaging Table 1. Short-cut estimates from range of
all available data. values in population (Snedecor and Cochran, 1964)
• For an approximate estimate of the
variance with only a limited number Number Weighting Number Weighting
of data, one can use the short-cut of points factor of points factor
estimates suggested by Snedecor 1 - 11 0.315
and Cochran (1964). The standard 2 0.886 12 0.307
deviation is obtained by multiplying
the range of the available values by 3 0.591 13 0.300
a weighting factor as shown in the 4 0.486 14 0.294
Table 1. The range is defined as the 5 0.430 15 0.288
difference between the largest and
smallest values in the data 6 0.395 16 0.283
population. 7 0.370 17 0.279
• For example, for five data points, 8 0.351 18 0.275
the standard deviation would be the 9 0.337 19 0.271
range multiplied by 0.43. The
approach is a good estimator for 10 0.325 20 0.268
symmetric data populations.
CIVL576/Zhang 8
 Probability concepts:
Linear regression
It is often the relation between two
parameters which is of relevance for a
calculation, for example correlation
between corrected cone penetration
resistance and undrained shear
strength from triaxial compression
tests via a cone factor, or variation of
undrained shear strength with depth.
For random variables, regression
analysis is used to establish the
statistics of the relationship. The
goodness of fit is given by how close
the correlation coefficient (ρ) is to
unity. Ang and Tang (1975) described
in detail the mechanics of linear
regression analysis for constant and
non-constant variance with depth.
CIVL576/Zhang 9
 Probability concepts: correlation
Consider random variables X and Y. The
joint second moment about the means
μX and μY is the covariance of X and Y;
that is,
Cov ( X , Y ) = E[( X − μ X )(Y − μY )]
= E ( XY ) − E ( X ) E (Y )

Cov (X, Y) = 0 if X and Y are

statistically independent.
The normalized covariance or
correlation coefficient is defined as

Cov( X , Y )
ρ=
σ X σY
The value of ρ ranges between -1 and
+1; that is, ρ2 ≤1.0.

CIVL576/Zhang 10
 Statistical treatment of test data:
Geostatistical methods
ƒ Sources of uncertainty and variability of soil properties
ƒ Review of probability concepts
ƒ Characterizing spatial variability of soil properties
ƒ Geostatistical analysis of CPT data
ƒ Site investigation strategy
ƒ Appendix I: Geostatistics (Fenton 1997)
ƒ Appendix II: Probabilistic methods in geotechnical engineering
(US National Research Council 1995)

CIVL576/Zhang 11
 Spatial variability of soil properties
• For any extensive volume of a natural soil layer, the characteristics
fluctuate spatially. The soil properties are then considered to be
controlled by a random process, with a random pattern of variation
and at times an overall trend. The variability may be small or large.
There is a greater tendency for the properties to be similar in value at
closely neighboring points than at widely spaced points. Soil
properties are expected to show dependence both laterally and with
depth. However, in the vertical direction, soil density and strength for
example depend on the effective overburden stress.
• The spatial variability of soil properties in situ within a given geologic
layer is often modeled by trend surfaces and residual variations about
the trend.

CIVL576/Zhang 12
CIVL576/Zhang 13
 • Consider a clay barrier of extent 40x40
m. If permeability tests were carried out
at sufficiently close spacing, the
permeability might appear as shown in
Fig. 4. The random field, X (x) is
characterized by:
ƒ its mean, μ(x). This may be spatially
constant, or may vary as a function
of x, the latter a feature of non-
stationary, also called non-
homogeneous, random fields,
ƒ its variance, σ2(x), which represents
the degree of scatter in the field
about its mean,
ƒ its correlation structure, ρ(x, x ’),
which gives the correlation
coefficient between X(x) and X(x ’)
for any two points x and x ’.

CIVL576/Zhang 14
 Estimating the mean and variance
• The task of data analysis and geostatistics is to deduce from a set of
data the appropriate parameters of a random field model of the soil
property in question. This generally means that values for the mean,
variance and correlation structure are to be found.
• Given a set of data Xi, its mean and variance can be estimated as
Eeq. (1)

• If a different set of observations of X are used, the estimated mean will

likely be different, that is itself is a random variable. If the Xi’s are
independent the variance of the mean decreases as the number of
observations n increases

CIVL576/Zhang 15
 • If Xi’s are perfectly corrected as in X1 = X2 = … = Xn, then

That is, there is no reduction in the variability of the estimate as n

increases.
• In general, the variance of the estimate is somewhere between σX2 and
σX2/n,

where ρij is the correlation coefficient between Xi and Xj and γ(T) is called
the variance function. The variance function lies between 0 and 1 and gives
the amount of variance reduction that takes place when X is averaged over
the sampling domain T = nΔx.
ƒ For highly correlated fields, the variance function tends to be close to 1,
ƒ while for poorly correlated fields, the variance function tends towards
Δx/T= 1/n .
CIVL576/Zhang 16
 Autocorrelation
• The variation of a soil property in space is
illustrated in Figure 5, as a function of a trend,
T(x), and residuals, εr(x). The residuals over a
large volume of soil are assumed to have zero
mean. The trend function is obtained by
regression analysis; the residuals are correlated,
unless the data are very widely spaced.
• The spatial variation of a soil property can be
modeled as
Y ( x) = T ( x) + ε r ( x)
where Y(x) = measurement at location x; T(x) =
trend component and εr(x) = residual. Figure 5. Spatial variability of soil
• The deviations about the trend tend to exhibit property
spatial correlation. The degree of spatial
correlation can be expressed through an
autocovariance function:
C (r ) = E[(Y (i ) − T (i )) • (Y ( j ) − T ( j ))] = E[ε ri • ε rj ]
where r is the vector of separation distance
between points i and j, E[.] is the expectation
operator.
CIVL576/Zhang 17
 Common
autocovariance
functions

CIVL576/Zhang 18
 • The properties of an extensive volume of natural soil layer inevitably
fluctuate. Soil properties tend to exhibit a strong spatial correlation
structure, which appears as a waviness about the trend. The
properties tend to be more similar in value at closely spaced points
than at widely spaced points.
• The larger the width, length or depth over which a parameter is
averaged, the more the fluctuations tend to cancel each other in the
process of “spatial averaging.”
• This correlation structure can be important both for improving
estimates at unsampled locations and for assessing the reliability of a
soil parameter.

CIVL576/Zhang 19
 • Normalized form of the autocovariance
function is known as the auto-correlation
function (or correlation coefficient):
C (r )
ρ(r ) =
C ( 0)
where C(0) is the autocovariance function
at r = 0.
• In the exponential models, the distance at
which the autocovariance function Cr =
σ2 e-(r/r0) decays to a value of 1/e is called
the autocovariance distance, r0. This
characteristic length describes the extent of
the spatial correlation. Fig. 7 shows how Figure 7. Influence of variance and
the covariance distance and the variance autovariance distance ro on variation of
influence the fluctuation of a soil property soil property
(de Groot and Baecher 1993)

CIVL576/Zhang 20
 Estimating the correlation structure
An estimator of the correlation structure of a one-dimensional random field
will be developed here. Consider the sequence of random variables {X1, X2,
…, Xn} sampled from X(x) at a sequence of locations separated by distance
Δx. For the following estimator, it is essential that the data be equi-spaced.
An unbiased estimator for the covariance, B(jΔx) between any two random
variables along x separated by the distance jΔx. For j = 0, 1, …, n–M-1 is
given by

where M is the number of unknowns used to estimate μX(x). If only one

unknown (e.g. permeability), then M = 1. The correlation coefficient is
then estimated as

where is the estimated variance.

A two-dimensional case is illustrated in an example.

CIVL576/Zhang 21
 Example 1: statistical analysis of permeability
Consider a set of permeability measurements made by infiltrometer on 2 ft. by 2 ft.
cells extracted from a rectangular test pad of poorly compacted clay, as shown in
Table 4.1. The test pad is of dimension 16 ft. by 16 ft. and the (x1, x2) coordinates
shown on the table correspond to the center of each 2 ft. square cell. All values are in
units of 10-7 cm/sec. A quick review of the data reveals first that it is highly variable
with Kmax/Kmin> 4000 and second that it tends from very high values at the left edge (
x1 = 1) to small values as x1 increases. There also appears to be a similar, but
somewhat less pronounced trend in the x2 direction, at least for larger values of x1.

CIVL576/Zhang 22
 Step 1: Data transformation
• Assume any apparent trends in the data are treated as simply part of a longer
scale fluctuation, or the field is assumed to be stationary in mean and variance.
• Permeability has shown by researchers to follow the lognormal distribution. So
the data are transformed to ln K. Using Eq. (1), the mean and variance are
estimated as

CIVL576/Zhang 23
 Step 2: Calculating correlation coefficients
Assumptions:
a) assume that the clay bed is isotropic, which appears physically reasonable.
Hence an isotropic correlation structure would be adopted which can be
estimated by averaging over the lag τ in any direction. For example, when τ = 2
feet the correlation can be estimated by averaging over all samples separated
by 2 ft. in any direction.
b) assume that the principal axes of anisotropy are aligned with the x1 and x2
coordinate axes and that the correlation function is separable. Now
is obtained by averaging in the two coordinate
directions separately and lag vectors not aligned with the coordinates need not
be considered. Because of the reduced number of samples contributing to each
estimate, the estimates themselves will be more variable.
c) assume that the correlation structure is more generally anisotropic. Lags in any
direction must be considered separately and certain directions and lags will
have very few data pairs from which to derive an estimate. This typically
requires a large amount of data.

Assumption (a) is preferred, but (b) will also be examined to judge the
applicability of the first assumption.
CIVL576/Zhang 24
 In assumption (b), the directional estimators are given by

where is the deviation in ln K about the mean; n1 and n2 are the

numbers of samples in the x1 and x2 direction respectively; Δt1 = Δt2 = 2 feet in this
example. The subscripts on X’ or ln K index first the x1 direction and second the x2
direction.
The isotropic correlation estimator of assumption (a) is obtained using

In which if n1 ≠ n2, then the (ni – j) appearing in the denominator must be

treated specially. Specifically for any j > ni, the ni- j term is set to zero.

CIVL576/Zhang 25
 Fig. 4.5 shows the estimated directional and isotropic correlation functions for the ln K
data. Note that at higher lags, the curves become quite erratic. This is typical since they
are based on fewer sample pairs as the lag increases. Also shown on the plot is a fitted
exponentially decaying correlation function. The scale of fluctuation, θ, is estimated to be
about 5 ft. in this case by simply passing the curve through the estimated correlation at
lag τ = 2 ft. Note that the estimated scale is quite sensitive to the mean. For example, if
the mean is known to be -12.0 rather than -13.86, then the estimated scale using this
data jumps to 15 ft. In effect, the estimated scale is quite uncertain; it is best used to
characterize the site at which the data was taken.

CIVL576/Zhang 26
 Example 2
• The autocorrelation is the basic reason for a
reduction in the scatter in the data (standard
deviation) as the averaging dimension
decreases. Figure 5.122 gives an example of
the autocorrelation structure of cone
penetration data at a depth of 9 m in a
dense sand. The scale of fluctuation, which is
a function of the shape of the autocorrelation
function, is directly related to the
autocorrelation distance, ro, and represents
the distance within which the soil property
shows strong correlation.
• In the example, the autocorrelation function
had the form:
− ( r / 37.5 ) 2
ρ(r ) = 0.986 e
where r and 37.5 are distance in metres. The
closeness of the factor 0.986 to unity
indicates that there was little measurement
noise in the recorded data.
CIVL576/Zhang 27
 Autocorrelation
distance r0

CIVL576/Zhang 28
 Statistical treatment of test data:
Geostatistical methods
ƒ Sources of uncertainty and variability of soil
properties
ƒ Review of probability concepts
ƒ Characterizing spatial variability of soil properties
ƒ Geostatistical analysis of CPT data
ƒ Site investigation strategy
ƒ Appendix I: Geostatistics (Fenton 1997)
ƒ Appendix II: Probabilistic methods in geotechnical
engineering (US National Research Council 1995)

CIVL576/Zhang 29
 Geostatistical analysis
• Properly accounting for the variability in soil properties when
predicting geotechnical performance may reduce substantially the
uncertainties in the soil parameters and therefore the uncertainties
associated with a design.
• One is never able to gather enough subsurface data to get an exact
picture of the variation of a soil property for an engineering structure.
One must therefore interpolate the soil properties within a large
volume, and sometimes one needs to extrapolate.
• Traditional interpretation methods used in geotechnical engineering
give little regard to the uncertainties associated with soil properties
and the fact that soil properties tend to exhibit a spatial correlation
structure.

CIVL576/Zhang 30
 Geostatisitcal analysis example (Nadim 1988)
- CPT testing for a shallow foundation (1)
In the neighborhood of a shallow
foundation, several CPT tests were run
at different times.

Fig 5.123 illustrates the locations of all

available soundings in the regional
investigation and Fig. 5.124 presents
the points selected for a local analysis
very close to the foundation. Fig.
5.125 gives examples of the qc results.
The soil profile consists of a top sand
(7-10 m) over a weaker, partly
laminated clay. In this case, the
location of the weaker layer and the
undrained shear strength for the
weaker material were the determinant
factor for the foundation. Using
minimum values would result in large
added costs.

CIVL576/Zhang 31
CIVL576/Zhang 32
 Figure 5.126 shows a comparison
of the contours of qc at a depth of
10 m, for the regional and the
local data points. Although the
scatter in the data is large, which
is to be expected in a layer where
both sand and clay can be
intermixed, the Kriging mean is
significantly higher than what
would have been assumed in
design at that depth (about 1.5 to
2 MPa).

Kriging is a stochastic interpolation

technique (Please refer to
Appendix II for detail).

CIVL576/Zhang 33
 • By illustrating the spatial distribution in
three-dimensions of the measured cone
resistance, as done in Figure 5.127 and
5.128, one gets a much better insight
into the possible variation in the cone
resistance beneath the foundation and
the most likely values.
• The plots show that there does not
seem to be a continuous layer with
weaker shear strength directly beneath
the structure. At this depth, the best fit
autocorrelation function has the
following exponential form:
− ( r / 37.5 ) 2
ρ(r ) = 0.986 e
The factor of unity indicates no noise in
the measurements. However, the
coefficient of variation for the data was
about 50%. Figure 5.129 illustrates the
autocorrelation function.

CIVL576/Zhang 34
 • Figure 5.128 gives a graphical
representation at each meter
between 7 and 12 m. It is
possible to make statistical
analyses at as many depths as
required for a problem.
• In this particular case study,
the geostatistical analysis
enabled the designers to adjust
the assumed position of the
clay layer below the depth that
had originally been assumed
and to use slightly higher cone
penetration resistance values,
and therefore higher shear
strength in design.
CIVL576/Zhang 35
 • The advantage of the Kriging
interpolation technique is that the
geotechnical parameters for the
analysis of a structure are more
clearly defined with respect to their
mean and variance. On the other
hand, to do such characterization a
relatively large quantity of data is
needed.
• Figure 5.131 illustrates how one
obtains the reduction factor to account
for spatial averaging. The reduction
factor is obtained from the square root
of the autocovariance function. Within
the scale of fluctuation, the reduction
in variance can vary by a factor as
much as 0.4 to 0.8. Techniques to do
spatial averaging exist and were
described by Vanmarcke (1977, 1983).
They should be applied, when a
sufficient quantity of data is available,
to obtain representative standard
deviations of soil variables.

CIVL576/Zhang 36
 Site investigation strategy
• Statistics can also be used to optimize soil investigation programs.
The optimizing strategy presents decision paths for the sampling, in
situ testing and laboratory testing of soil. Initially, the site
investigation strategy was solved manually with flow charts. The
newer versions of the system are solved on personal computers with
a knowledge-based system to make decisions when required,
providing recommendations for optimum sampling, in situ testing, and
laboratory testing programs (Haarvik and Lacasse, 1990).
• The uncertainty in a geotechnical calculation is often related to the
uncertain presence of an "anomaly", for example boulders, soft clay
pockets or even a drainage layer. Statistical approaches can be used
to combine objective measurements to subjective estimates or
information from other sources. Problems such as presence of gas or
weaker layer, the probability of not detecting a drainage layer or an
anomaly, the increased safety margin against presence of anomaly for
each additional in-situ test, can also be solved by statistical methods.
CIVL576/Zhang 37
 Summary
• Basic statistics represent a useful way to establish mean and variance. In
geotechnical problems that involve large soil volumes, the "spatially averaged"
properties govern foundation analysis, as local fluctuations average over a large
soil volume. One should then use a site description strategy, including
ƒ identification of the correlation structure and autocorrelation function of the
soil property with given geologic units, and
ƒ use of the stochastic interpolation technique (e.g. Kriging) to estimate the
soil property at the location of interest. The added knowledge on the
uncertainties in the soil parameters and from the subsequent reliability
analyses accounting for these uncertainties should lead to safer designs.
• Geostatistics provide useful complementary information to help guide on the
interpretation of soil profiles, stratigraphy and horizontal and vertical variability
in the soil properties.
• The analysis will also give an estimate of the noise in the measurements and
help discern actual in situ trends from anomalies. The approach is well suited for
the interpretation of CPT data since a lot of data points are generated, and
hopefully it will become more usual to run a large quantity of profilings.
• The application of geostatistics requires that a large quantity of data be
available for generating the autocorrela-tion function and obtaining reliable
mean and standard deviation or variance values.
• Statistical methods can also be used to plan site investigation programs and for
optimizing the location and number of additional tests.
CIVL576/Zhang 38