Chapter Eighteen

Discriminant Analysis
 18-2

Chapter Outline
1) Overview
2) Basic Concept
3) Relation to Regression and ANOVA
4) Discriminant Analysis Model
5) Statistics Associated with Discriminant Analysis
6) Conducting Discriminant Analysis
i. Formulation
ii. Estimation
iii. Determination of Significance
iv. Interpretation
v. Validation
 18-3

Chapter Outline
7) Multiple Discriminant Analysis
i. Formulation
ii. Estimation
iii. Determination of Significance
iv. Interpretation
v. Validation
8) Stepwise Discriminant Analysis
9) Internet and Computer Applications
10) Focus on Burke
11) Summary
12) Key Terms and Concepts
 Similarities and Differences between ANOVA,
18-4

Regression, and Discriminant Analysis

Table 18.1

ANOVA REGRESSION DISCRIMINANT ANALYSIS

Similarities
Number of One One One
dependent
variables
Number of
independent Multiple Multiple Multiple
variables

Differences
Nature of the
dependent Metric Metric Categorical
variables
Nature of the
independent Categorical Metric Metric
variables
 18-5

Discriminant Analysis
Discriminant analysis is a technique for analyzing data when
the criterion or dependent variable is categorical and the
predictor or independent variables are interval in nature.

The objectives of discriminant analysis are as follows:

 Development of discriminant functions, or linear

combinations of the predictor or independent variables, which

will best discriminate between the categories of the criterion or
dependent variable (groups).
 Examination of whether significant differences exist among the

groups, in terms of the predictor variables.

 Determination of which predictor variables contribute to most of

the intergroup differences.

 Classification of cases to one of the groups based on the values

of the predictor variables.

 Evaluation of the accuracy of classification.
 18-6

Discriminant Analysis
 When the criterion variable has two categories, the
technique is known as two-group discriminant analysis.
 When three or more categories are involved, the technique
is referred to as multiple discriminant analysis.
 The main distinction is that, in the two-group case, it is
possible to derive only one discriminant function. In
multiple discriminant analysis, more than one function may
be computed. In general, with G groups and k predictors,
it is possible to estimate up to the smaller of G - 1, or k,
discriminant functions.
 The first function has the highest ratio of between-groups
to within-groups sum of squares. The second function,
uncorrelated with the first, has the second highest ratio,
and so on. However, not all the functions may be
statistically significant.
 18-7

Discriminant Analysis Model

The discriminant analysis model involves linear combinations of
the following form:

D = b0 + b1X1 + b2X2 + b3X3 + . . . + bkXk

where
 
D = discriminant score
b 's = discriminant coefficient or weight
X 's = predictor or independent variable

 The coefficients, or weights (b), are estimated so that the

groups differ as much as possible on the values of the
discriminant function.
 This occurs when the ratio of between-group sum of squares to
within-group sum of squares for the discriminant scores is at a
maximum.
 18-8

Statistics Associated with Discriminant Analysis

 Canonical correlation. Canonical correlation
measures the extent of association between the
discriminant scores and the groups. It is a measure of
association between the single discriminant function
and the set of dummy variables that define the group
membership.
 Centroid. The centroid is the mean values for the
discriminant scores for a particular group. There are as
many centroids as there are groups, as there is one for
each group. The means for a group on all the functions
are the group centroids.
 Classification matrix. Sometimes also called
confusion or prediction matrix, the classification matrix
contains the number of correctly classified and
misclassified cases.
 18-9

Statistics Associated with Discriminant Analysis

 Discriminant function coefficients. The
discriminant function coefficients (unstandardized)
are the multipliers of variables, when the variables
are in the original units of measurement.
 Discriminant scores. The unstandardized
coefficients are multiplied by the values of the
variables. These products are summed and added to
the constant term to obtain the discriminant scores.
 Eigenvalue. For each discriminant function, the
Eigenvalue is the ratio of between-group to within-
group sums of squares. Large Eigenvalues imply
superior functions.
 18-10

Statistics Associated with Discriminant Analysis

 F values and their significance. These are
calculated from a one-way ANOVA, with the grouping
variable serving as the categorical independent
variable. Each predictor, in turn, serves as the metric
dependent variable in the ANOVA.
 Group means and group standard deviations.
These are computed for each predictor for each
group.
 Pooled within-group correlation matrix. The
pooled within-group correlation matrix is computed
by averaging the separate covariance matrices for all
the groups.
 18-11

Statistics Associated with Discriminant Analysis

 Standardized discriminant function coefficients. The
standardized discriminant function coefficients are the
discriminant function coefficients and are used as the multipliers
when the variables have been standardized to a mean of 0 and a
variance of 1.
 Structure correlations. Also referred to as discriminant
loadings, the structure correlations represent the simple
correlations between the predictors and the discriminant function.
 Total correlation matrix. If the cases are treated as if they
were from a single sample and the correlations computed, a total
correlation matrix is obtained.
 Wilks' . Sometimes also called the U statistic, Wilks' for
each predictor
 is the ratio of the within-group sum of squares
 to
the total sum of squares. Its value varies between 0 and 1.
Large values of (near 1) indicate that group means do not
seem to be different.
 Small values of (near 0) indicate that
the group means seem to be different. 
 18-12

Conducting Discriminant Analysis

Fig. 18.1

Formulate the Problem

Estimate the Discriminant Function Coefficients

Determine the Significance of the Discriminant Function

Interpret the Results

Assess Validity of Discriminant Analysis

Conducting Discriminant Analysis 18-13

Formulate the Problem

 Identify the objectives, the criterion variable, and the
independent variables.
 The criterion variable must consist of two or more
mutually exclusive and collectively exhaustive categories.
 The predictor variables should be selected based on a
theoretical model or previous research, or the experience
of the researcher.
 One part of the sample, called the estimation or analysis
sample, is used for estimation of the discriminant
function.
 The other part, called the holdout or validation sample,
is reserved for validating the discriminant function.
 Often the distribution of the number of cases in the
analysis and validation samples follows the distribution in
the total sample.
 18-14

Information on Resort Visits: Analysis Sample

Table 18.2
Annual Attitude Importance Household Age of Amount
Resort Family Toward Attached Size Head of Spent on
No. Visit Income Travel to Family Household
Family
($000) Vacation Vacation

1 1 50.2 5 8 3 43 M (2)
2 1 70.3 6 7 4 61 H (3)
3 1 62.9 7 5 6 52 H (3)
4 1 48.5 7 5 5 36 L (1)
5 1 52.7 6 6 4 55 H (3)
6 1 75.0 8 7 5 68 H (3)
7 1 46.2 5 3 3 62 M (2)
8 1 57.0 2 4 6 51 M (2)
9 1 64.1 7 5 4 57 H (3)
10 1 68.1 7 6 5 45 H (3)
11 1 73.4 6 7 5 44 H (3)
12 1 71.9 5 8 4 64 H (3)
13 1 56.2 1 8 6 54 M (2)
14 1 49.3 4 2 3 56 H (3)
15 1 62.0 5 6 2 58 H (3)
 18-15

Information on Resort Visits: Analysis Sample

Table 18.2 cont.
Annual Attitude Importance Household Age of Amount
Resort Family Toward Attached Size Head of Spent on
No. Visit Income Travel to Family
Household Family
($000) Vacation Vacation

16 2 32.1 5 4 3 58 L (1)
17 2 36.2 4 3 2 55 L (1)
18 2 43.2 2 5 2 57 M (2)
19 2 50.4 5 2 4 37 M (2)
20 2 44.1 6 6 3 42 M (2)
21 2 38.3 6 6 2 45 L (1)
22 2 55.0 1 2 2 57 M (2)
23 2 46.1 3 5 3 51 L (1)
24 2 35.0 6 4 5 64 L (1)
25 2 37.3 2 7 4 54 L (1)
26 2 41.8 5 1 3 56 M (2)
27 2 57.0 8 3 2 36 M (2)
28 2 33.4 6 8 2 50 L (1)
29 2 37.5 3 2 3 48 L (1)
30 2 41.3 3 3 2 42 L (1)
 Information on Resort Visits:
18-16

Holdout Sample
Table 18.3

Annual Attitude Importance Household Age of Amount

Resort Family Toward Attached Size Head of Spent on
No. Visit Income Travel to Family
Household Family
($000) Vacation Vacation

1 1 50.8 4 7 3 45 M(2)
2 1 63.6 7 4 7 55 H (3)
3 1 54.0 6 7 4 58 M(2)
4 1 45.0 5 4 3 60 M(2)
5 1 68.0 6 6 6 46 H (3)
6 1 62.1 5 6 3 56 H (3)
7 2 35.0 4 3 4 54 L (1)
8 2 49.6 5 3 5 39 L (1)
9 2 39.4 6 5 3 44 H (3)
10 2 37.0 2 6 5 51 L (1)
11 2 54.5 7 3 3 37 M(2)
12 2 38.2 2 2 3 49 L (1)
Conducting Discriminant Analysis 18-17

Estimate the Discriminant Function Coefficients

 The direct method involves estimating the
discriminant function so that all the predictors are
included simultaneously.
 In stepwise discriminant analysis, the predictor
variables are entered sequentially, based on their
ability to discriminate among groups.
 18-18

Results of Two-Group Discriminant Analysis

Table 18.4
GROUP MEANS
VISIT INCOME TRAVEL VACATION HSIZE AGE
1 60.52000 5.40000 5.80000 4.33333 53.73333
2 41.91333 4.33333 4.06667 2.80000 50.13333
Total 51.21667 4.86667 4.9333 3.56667 51.93333
Group Standard Deviations
1 9.83065 1.91982 1.82052 1.23443 8.77062
2 7.55115 1.95180 2.05171 .94112 8.27101
Total 12.79523 1.97804 2.09981 1.33089 8.57395
Pooled Within-Groups Correlation Matrix
INCOME TRAVEL VACATION HSIZE AGE
INCOME 1.00000
TRAVEL 0.19745 1.00000
VACATION 0.09148 0.08434 1.00000
HSIZE 0.08887 -0.01681 0.07046 1.00000
AGE - 0.01431 -0.19709 0.01742 -0.04301 1.00000
Wilks' (U-statistic) and univariate F ratio with 1 and 28 degrees of freedom
Variable Wilks' F Significance
INCOME 0.45310 33.800 0.0000
TRAVEL 0.92479 2.277 0.1425
VACATION 0.82377 5.990 0.0209
HSIZE 0.65672 14.640 0.0007
AGE 0.95441 1.338 0.2572
Contd.
 18-19

Results of Two-Group Discriminant Analysis

Table 18.4 cont.
CANONICAL DISCRIMINANT FUNCTIONS
% of Cum Canonical After Wilks'
Function Eigenvalue Variance % Correlation Function  Chi-square df Significance
: 0 0 .3589 26.130 5 0.0001
1* 1.7862 100.00 100.00 0.8007 :
* marks the 1 canonical discriminant functions remaining in the analysis.
Standard Canonical Discriminant Function Coefficients
FUNC 1
INCOME 0.74301
TRAVEL 0.09611
VACATION 0.23329
HSIZE 0.46911
AGE 0.20922
Structure Matrix:
Pooled within-groups correlations between discriminating variables & canonical discriminant functions
(variables ordered by size of correlation within function)
FUNC 1
INCOME 0.82202
HSIZE 0.54096
VACATION 0.34607
TRAVEL 0.21337
AGE 0.16354
Contd.
 18-20

Results of Two-Group Discriminant Analysis

Table 18.4 cont.

Unstandardized Canonical Discriminant Function Coefficients

FUNC 1
INCOME 0.8476710E-01
TRAVEL 0.4964455E-01
VACATION 0.1202813
HSIZE 0.4273893
AGE 0.2454380E-01
(constant) -7.975476
Canonical discriminant functions evaluated at group means (group centroids)
Group FUNC 1
1 1.29118
2 -1.29118
Classification results for cases selected for use in analysis
Predicted Group Membership
Actual Group No. of Cases 1 2
Group 1 15 12 3
80.0% 20.0%
Group 2 15 0 15
0.0% 100.0%
Percent of grouped cases correctly classified: 90.00%
Contd.
 18-21

Results of Two-Group Discriminant Analysis

Table 18.4 cont.

Classification Results for cases not selected for use in the analysis (holdout sample)
Predicted Group Membership
Actual Group No. of Cases 1 2
Group 1 6 4 2
66.7% 33.3%
Group 2 6 0 6
0.0% 100.0%
Percent of grouped cases correctly classified: 83.33%.
 18-22
Conducting Discriminant Analysis
Determine the Significance of Discriminant Function

 The null hypothesis that, in the population, the

means of all discriminant functions in all groups are
equal can be statistically tested.
 In SPSS this test is based on Wilks'  . If several
functions are tested simultaneously (as in the case of
multiple discriminant analysis), the Wilks'  statistic
is the product of the univariate for each function.
The significance level is estimated based on a chi-
square transformation of the statistic.
 If the null hypothesis is rejected, indicating
significant discrimination, one can proceed to
interpret the results.
Conducting Discriminant Analysis 18-23

Interpret the Results

 The interpretation of the discriminant weights, or coefficients, is
similar to that in multiple regression analysis.
 Given the multicollinearity in the predictor variables, there is no
unambiguous measure of the relative importance of the predictors
in discriminating between the groups.
 With this caveat in mind, we can obtain some idea of the relative
importance of the variables by examining the absolute magnitude of
the standardized discriminant function coefficients.
 Some idea of the relative importance of the predictors can also be
obtained by examining the structure correlations, also called
canonical loadings or discriminant loadings. These simple
correlations between each predictor and the discriminant function
represent the variance that the predictor shares with the function.
 Another aid to interpreting discriminant analysis results is to develop
a characteristic profile for each group by describing each group
in terms of the group means for the predictor variables.
Conducting Discriminant Analysis 18-24

Access Validity of Discriminant Analysis

 Many computer programs, such as SPSS, offer a leave-one-
out cross-validation option.
 The discriminant weights, estimated by using the analysis
sample, are multiplied by the values of the predictor
variables in the holdout sample to generate discriminant
scores for the cases in the holdout sample. The cases are
then assigned to groups based on their discriminant scores
and an appropriate decision rule. The hit ratio, or the
percentage of cases correctly classified, can then be
determined by summing the diagonal elements and dividing
by the total number of cases.
 It is helpful to compare the percentage of cases correctly
classified by discriminant analysis to the percentage that
would be obtained by chance. Classification accuracy
achieved by discriminant analysis should be at least 25%
greater than that obtained by chance.
 18-25

Results of Three-Group Discriminant Analysis

Table 18.5

Group Means
AMOUNT INCOME TRAVEL VACATION HSIZE AGE

1 38.57000 4.50000 4.70000 3.10000 50.30000

2 50.11000 4.00000 4.20000 3.40000 49.50000
3 64.97000 6.10000 5.90000 4.20000 56.00000
Total 51.21667 4.86667 4.93333 3.56667 51.93333

Group Standard Deviations

1 5.29718 1.71594 1.88856 1.19722 8.09732

2 6.00231 2.35702 2.48551 1.50555 9.25263
3 8.61434 1.19722 1.66333 1.13529 7.60117
Total 12.79523 1.97804 2.09981 1.33089 8.57395

Pooled Within-Groups Correlation Matrix

INCOME TRAVEL VACATION HSIZE AGE

INCOME 1.00000
TRAVEL 0.05120 1.00000
VACATION 0.30681 0.03588 1.00000
HSIZE 0.38050 0.00474 0.22080 1.00000
AGE -0.20939 -0.34022 -0.01326 -0.02512 1.00000
Contd.
 18-26

Results of Three-Group Discriminant Analysis

Table 18.5 cont.

Wilks' (U-statistic) and univariate F ratio with 2 and 27 degrees of freedom.

Variable Wilks' Lambda F Significance

INCOME 0.26215 38.00 0.0000

TRAVEL 0.78790 3.634 0.0400
VACATION 0.88060 1.830 0.1797
HSIZE 0.87411 1.944 0.1626
AGE 0.88214 1.804 0.1840

CANONICAL DISCRIMINANT FUNCTIONS

% of Cum Canonical After Wilks'

Function Eigenvalue Variance % Correlation Function  Chi-square df Significance
: 0 0.1664 44.831 10 0.00
1* 3.8190 93.93 93.93 0.8902 : 1 0.8020 5.517 4 0.24

2* 0.2469 6.07 100.00 0.4450 :

* marks the two canonical discriminant functions remaining in the analysis.

Standardized Canonical Discriminant Function Coefficients

FUNC 1 FUNC 2
INCOME 1.04740 -0.42076
TRAVEL 0.33991 0.76851
VACATION -0.14198 0.53354
HSIZE -0.16317 0.12932
AGE 0.49474 0.52447 Contd.
 18-27

Results of Three-Group Discriminant Analysis

Table 18.5 cont.
Structure Matrix:
Pooled within-groups correlations between discriminating variables and canonical discriminant
functions (variables ordered by size of correlation within function)

FUNC 1 FUNC 2
INCOME 0.85556* -0.27833
HSIZE 0.19319* 0.07749
VACATION 0.21935 0.58829*
TRAVEL 0.14899 0.45362*
AGE 0.16576 0.34079*

Unstandardized canonical discriminant function coefficients

FUNC 1 FUNC 2
INCOME 0.1542658 -0.6197148E-01
TRAVEL 0.1867977 0.4223430
VACATION -0.6952264E-01 0.2612652
HSIZE -0.1265334 0.1002796
AGE 0.5928055E-01 0.6284206E-01
(constant) -11.09442 -3.791600

Canonical discriminant functions evaluated at group means (group centroids)

Group FUNC 1 FUNC 2
1 -2.04100 0.41847
2 -0.40479 -0.65867
3 2.44578 0.24020 Contd.
 18-28

Results of Three-Group Discriminant Analysis

Table 18.5 cont.
Classification Results:
Predicted Group Membership
Actual Group No. of Cases 1 2 3

Group 1 10 9 1 0
90.0% 10.0% 0.0%

Group 2 10 1 9 0
10.0% 90.0% 0.0%

Group 3 10 0 2 8
0.0% 20.0% 80.0%
Percent of grouped cases correctly classified: 86.67%

Classification results for cases not selected for use in the analysis
Predicted Group Membership
Actual Group No. of Cases 1 2 3

Group 1 4 3 1 0
75.0% 25.0% 0.0%

Group 2 4 0 3 1
0.0% 75.0% 25.0%

Group 3 4 1 0 3
25.0% 0.0% 75.0%
Percent of grouped cases correctly classified: 75.00%
 18-29

All-Groups Scattergram
Fig. 18.2

Across: Function 1
Down: Function 2

4.0
1 1
1 *1 3
23 3 *3 3
1 1 12 * 3 3
0.0 1 1 2 2
3
1 2 2
2
-4.0

* indicates a group
centroid

-6.0 -4.0 -2.0 0.0 2.0 4.0 6.0

 18-30

Territorial Map
Fig. 18.3
13
13
13 Across: Function 1
8.0 Down: Function 2
13
13 * Indicates a
13
group centroid
13
4.0 113
112 3
112233
*1 1 1 2 2 2 2 3 3 *
1 1 2 2* 223
0.0 1122 233
1122 2233
11122 223
233
-4.0 11222
1122 223
11122 233
11122 2233
1122 223
-8.0 11122 233

-8.0 -6.0 -4.0 -2.0 0.0 2.0 4.0 6.0 8.0

 18-31

Stepwise Discriminant Analysis

 Stepwise discriminant analysis is analogous to
stepwise multiple regression (see Chapter 17) in that
the predictors are entered sequentially based on their
ability to discriminate between the groups.
 An F ratio is calculated for each predictor by
conducting a univariate analysis of variance in which
the groups are treated as the categorical variable and
the predictor as the criterion variable.
 The predictor with the highest F ratio is the first to
be selected for inclusion in the discriminant function,
if it meets certain significance and tolerance criteria.
 A second predictor is added based on the highest
adjusted or partial F ratio, taking into account the
predictor already selected.
 18-32

Stepwise Discriminant Analysis

 Each predictor selected is tested for retention based
on its association with other predictors selected.
 The process of selection and retention is continued
until all predictors meeting the significance criteria
for inclusion and retention have been entered in the
discriminant function.
 The selection of the stepwise procedure is based on
the optimizing criterion adopted. The Mahalanobis
procedure is based on maximizing a generalized
measure of the distance between the two closest
groups.
 The order in which the variables were selected also
indicates their importance in discriminating between
the groups.
 18-33

SPSS Windows
The DISCRIMINANT program performs both two-
group and multiple discriminant analysis. To select
this procedure using SPSS for Windows click:

Analyze>Classify>Discriminant …