Huberty (1989) stated that discriminant analysis (DA) includes a set of response variables and a set of one or more grouping or nominally scaled variables. Klecka (1980, p. 8) outlined the basic prerequisites for conducting a discriminant analysis, i.e., "two or more groups exist which we presume differ on several variables and that those variables can be measured at the interval or ratio level." There is no limit to the types of variables that can be employed, but problems with interpretation may result (Nunnally & Bernstein, 1994). Klecka (1980, p. 8) noted that "discriminant analysis will then help us analyze the differences between the groups and/or provide us with a means to assign (classify) any case into the groups which it most closely resembles."

As Huberty's description and Klecka's prerequisites in the above paragraph imply, discriminant analysis has two sets of techniques based on the purpose of the analysis, i.e., predictive discriminant analysis and descriptive discriminant analysis. "When groups of units are known in advance and the purpose of the research is either to describe group differences [DDA] or to predict group membership [PDA] on the basis of response variable measures, discriminant analysis techniques are appropriate" (Huberty, 1994). Alternatively, Stevens (1996) described the distinction between PDA and DDA in the following way: "in predictive discriminant analysis the focus is on classifying subjects into one of several groups, whereas in descriptive discriminant analysis the focus is on revealing major differences among the groups" (Stevens, 1996).

Discriminant analysis has been described by some researchers as similar to multiple regression (MR) analysis (Gall, Borg, & Gall, 1996) inasmuch as it is an adaptation of regression analysis techniques (Kachigan, 1986). In fact, anyone who is familiar with the basic goals and techniques of multiple regression can easily understand the association between multiple regression and discriminant analysis. Unfortunately, as Kachigan (1986, p. 375) pointed out, DA is sometimes used in instances where regression analysis is a "more appropriate and powerful technique." In these cases the continuous criterion variable is dichotomized in order to lend the analysis to DA procedures. This practice of squandering variance information, by dichotomization or polychotomization of continuous variables, has been strongly criticized in the literature (Kerlinger, 1986; Thompson, 1994).

Despite the close association between DA and MR, it is important to note that some researchers have recognized that all parametric procedures can be derived from the same linear model which involves the use of least squares weights (Cohen, 1968; Knapp, 1978). As Knapp (1978) noted, "virtually all of the commonly encountered parametric tests of significance can be treated as special cases of canonical correlation analysis, which is the general procedure for investing differences between two sets of variables" (p. 410). Thompson (1988) pointed out that every parametric procedure involves the creation of a synthetic score(s) for each individual on some latent construct. In discriminant analysis the synthetic scores are the discriminant scores created with the discriminant function coefficients (Pedhazur, 1982).

Several researchers (Huberty, 1989; Huberty, 1994; and Thompson, 1989) have noted the common use of stepwise analyses. According to Thompson (1989, p146), "stepwise analytic methods may be among the most popular research practices employed in both substantive and validity research." However, some of these same researchers as well as others (cf. Snyder, 1991) have advanced strong arguments against the use of stepwise methodologies.

The two types of discriminant analysis, i.e., PDA and DDA, have different histories of development. According to Huberty (1994), "discriminant analysis for the first three or four decades focused on the prediction of group membership," PDA, whereas DDA usage did not appear until the 1960s and "its use has been very limited in applied research settings over the past two decades." PDA and DDA are multivariate analyses that have important differences in both their general application and when used in conjunction with stepwise methodology.

  Predictive discriminant analysis (PDA), or "classification" as it is sometimes called, generally includes "a set of predictor variables and one criterion variable, the latter being a grouping variable with two or more levels, that is, there are two or more groups" (Huberty & Barton, 1989, p. 158). Predictive discriminant analysis is similar to multiple regression analysis except that PDA is used when the criterion variable is categorical and nominally scaled. As in multiple regression, in PDA a set of rules is formulated which consists of as many linear combinations of predictors as there are categories, or groups (Huberty, 1994). The equation in PDA uses a person's scores on the predictor variables to predict the category to which the individual belongs (Gall, Borg, & Gall, 1996, p. 441).

In predictive discriminant analysis each object will have a single score on the discriminant function in place of its scores on the various predictor variables. At the same time a cutoff score will be determined such that when the criterion groups are compared with respect to the discriminant scores the errors of classification are minimized (Kachigan, 1986, p. 365). Table 3 provides an example of a classification table used to report results from an application of a prediction rule. This heuristic provides information about the accuracy of the prediction rule, i.e., "the hit rates" or correct classifications. The overall percentage of correct classifications is 83.3%. The percentage of correct classifications must be judged against chance probabilities. Are our results better than chance? Yes, chance probabilities here would result in a 56% accurate classification rate.

DDA includes a collection of techniques involving two or more criterion variables and a set of one or more grouping variables, each with two or more levels, whose effects are assessed through MANOVA. "Whereas in predictive discriminant analysis (PDA) the multiple response variables play the role of predictor variables, in descriptive discriminant analysis (DDA) they are viewed as outcome variables and the grouping variable(s) as the explanatory variable(s). That is, the roles of the two types of variables involved in a multivariate, multigroup setting in DDA are reversed from the roles in PDA" (Huberty, 1994, p. 30). In DDA the total "between-groups" association in MANOVA is broken down into additive pieces through the use of uncorrelated linear combinations of the original variables (discriminant functions) (Stevens, 1996, p. 261).

According to Kerlinger & Pedhazur (1973, p. 337) "the discriminant function is a regression equation with a dependent variable that represents group membership." The aforementioned relationship between multiple regression and descriptive discriminant analysis is clearly illustrated in the two-group, or dichotomous grouping variable case, i.e., regression and DDA yield the same results. As can be seen from the heuristic example in Table 1, lambda at a given step equals 1 - R² and, conversely, R² equals 1 - lambda. Once the analysis changes to a DDA with more than 2-groups, the calculations become more complex and are no longer directly analogous to regression results.

Again, DDA is associated with MANOVA. When the results of the omnibus MANOVA effects has been shown to be generalizable, DDA can be used to describe and interpret these effects. The linear composites (linear discriminant functions, LDFs) can be used to identify outcome variable "constructs (or latent variables) that underlie the group differences, that is, that underlie the grouping variable effect" (Huberty, 1994, p. 206). Huberty (1994) stated that "the predominant method of identifying latent constructs in multivariate analyses--this includes factor analysis and canonical correlation--is to examine correlations between linear composite scores and scores on the individual variables in the composite. These LDF-variable correlations are often called structurer's" (p. 209). Interpretation of the latent construct underlying these functions is largely based on these structure r's (Huberty & Barton, 1989).

PDA is appropriate when the researcher is interested in assigning units (individuals) to groups based on composite scores on several predictor variables, (i.e., LCFs). The accuracy of such prediction can be assessed by examining "hit rates" as against chance, for example. The most basic question answered by PDA is "given the individuals scores on several predictor variables, which group represents their true membership group?" Again, the focus of PDA is prediction and the accuracy of hit rates. As Huberty and Barton (1989) noted with respect to PDA, "One is basically interested in determining a classification rule and assessing its accuracy"

History and Introduction

According to Huberty (1989), "stepwise analysis is believed to have been first advanced by Efroymson (1960), and is fully described by Draper and Smith (1981, chap. 6) and Jennrich (1977a, 1977b)." Several variants of stepwise methods are available through the statistical packages, e.g., forward selection, backward elimination, forward stepwise, and backward stepwise analysis. However, the default settings usually result in a forward selection analysis (Huberty, 1994, p. 261). Stepwise methodologies have enjoyed popular usage, especially in educational and psychological research settings. Huberty (1994) noted the widespread use of stepwise methods in empirically based journal articles. Thompson (1989) suggested that "stepwise analytic methods may be among the most popular research practices employed in both substantive and validity research" (p. 146).

Huberty (1989, p. 43) stated that three popular computer software packages, i.e., BMDP. SAS, and SPSS, include programs to conduct a "stepwise multiple regression analysis" and a "stepwise discriminant analysis." According to Stevens (1973; as cited in Huberty, 1989, p. 43), "although regression analysis and discriminant analysis problems are, without a doubt, the most popular contexts for the use of step-type computational algorithms, these approaches have also been suggested in multivariate analysis of variance" and in "canonical correlation analysis" (Thompson, 1984, pp. 47-51; Thorndike & Weiss, 1983; as cited in Huberty, 1989, p. 43).

Thompson (1995) remarked that "degrees of freedom in statistical analyses reflect the number of unique pieces of information present for a given research situation. These degrees of freedom constrain the number of inquiries we may direct at our data and are the currency we spend in analysis" (p. 526). In any computerized stepwise procedure the pre-set degrees of freedom are "one" for each variable included in the analysis. Thompson (1994) drew the analogy of the pre-set degrees of freedom as coins that we can spend to explore out data, or rather, we are charged one degree of freedom for every predictor variable used. However, at each step all predictors from the original variable set are considered for inclusion. So, at each step the correct number of degrees of freedom should be the same as the total number of variables from the predictor set. If the original number of predictor variables was ten than the correct "charge" is ten.

In the statistical test of significance, there are three calculations for degrees of freedom, i.e., explained, unexplained, and total. The computer packages calculate the correct "total" degrees of freedom (n-1). In regression the "explained" degrees of freedom are erroneously entered as the number of predictor variables (i.e., pv). Therefore, in regression the degrees of freedom "unexplained" (1-pv) are necessarily computed incorrectly (Thompson, 1995). Thompson provided a clear illustration of this type of error within a regression context in that same journal article.

Thompson's example involves data from 101 subjects on dependent variable, ("Y"), and 50 predictor variables. As can be seen from Table 4, the degrees of freedom computed by the computer packages (Analysis 1) yield a statistically significant ("=.05) result. However, the correct degrees of freedom are given in Analysis 2. As Thompson noted, "If the five entered predictor variables had been randomly selected, an explained degree of freedom of 5 might be arguably correct" (p. 527). The error is built into computer programs that do discriminant analyses.

  Numerous researchers have pointed out the benefits of interpreting structure type coefficients in both regression analysis (Bowling, 1993; Daniel, 1990; Perry, 1990; Thompson, 1992; Thompson & Borrello, 1985) and discriminant analysis (Pedhazur, 1982). In the present example, Table 6, the structure matrix reveals that variable Y₃, on function one, and variable Y₂, on function two, also contain much explanatory ability, or ability to account for variance. Therefore, the explanatory ability of variables Y₄ and Y₃ on the first function, and variables Y₁ and Y₂ on the second function, are very similar. The differences between Y₄ and Y₃, or between Y₁ and Y₂, may be due to sampling error.

  Third, the fact that stepwise methods do not identify the best predictor set of a given size is also problematic. Stepwise methods do not necessarily identify the best predictor set of a given size (Huberty, 1994; Thompson, 1995), even for the sample data being analyzed. The true best set (a) may yield considerably higher effect sizes and (b) may even include none of the variables selected by the stepwise algorithm.

Thompson (1995), using a stepwise regression example, described how stepwise procedures do not select the best set of predictor variables of size q. For example, five predictors entered in five steps of forward entry will not typically answer the question "What is the best set of q = 5 predictors?"

  Huberty and Wisenbaker (1992) indicated that the aforementioned computer programs (BMDP, SAS, and SPSS) provide most of the quantitative information needed to interpret a PDA or a DDA. In the computer programs discussed earlier in this paper, BMDP, SAS, and SPSS, PDA is used to classifying subjects into one of several groups, whereas DDA is designed to reveal major differences among groups. Stepwise methodologies can be used with either form of DA, but is typically used in a DDA context. The major differences between DDA and PDA are "revealed through the discriminant functions (Stevens, 1996). Three statistical packages, BMDP, SAS, and SPSS all perform a stepwise discriminant analysis (also stepwise regression analysis). Huberty (1994, p. 261) stated that "when it is claimed that a "stepwise ____ analysis" was run, more likely than not it was a forward stepwise analysis using default values for variable delection, which usually simply results in a forward analysis." 

  The statistical packages named above have stepwise discriminant analysis programs with built-in criteria for stepping that relate to group separation. In a forward analysis, variables are selected at each step such that group separation is increased the most. Therefore, inherent problems aside, stepwise methods would appear to be more appropriate in a MANOVA/DDA context where group separation is the focus of the discriminant analysis (Huberty, 1994, p. 261). Within this context, methods that increase the separation of groups by providing information about the importance of variables, an erroneous enticement offered by stepwise methodologies, would be valuable.

  Stepwise methods in a PDA context, where group membership prediction is the point of the analysis, would only be considered in "very restrictive situations" (Huberty, 1994, p. 261). According to Huberty (1989, p. 166), " it has not been shown that package stepwise results are relevant for a predictive discriminant analysis." Since PDA is concerned with hit rates and accuracy of classification, any reasonable PDA stepwise procedures must focus on maximizing hit rates. The widely used computer packages do not have stepwise algorithms that do this.

  The problems inherent with stepwise methodologies as outlined above are serious. Researchers such as Huberty (1989) do not "espouse the use of stepwise analyses" (p. 48). Huberty recognized that reducing the number of variables is sometimes warranted, as a preliminary analysis, e.g., to reduce the number of response variables to a manageable size. He suggested that variables be discarded when they do not provide predictive validity, for example, those that have contributed little to predictive validity in previous studies, variables highly correlated with other variables, and variables that are judged not relevant to the present study.

There are methods to determine the best subset of variables of size q. There are different ways to address the problem; however, perhaps the best solution is to use an "all-possible-subsets" approach (Huberty, 1989; Thompson, 1995). More specifically, researchers could conduct an all possible subsets of each size in order to determine the best subset of any given size. Computer programs are available that do this painlessly.

  Despite the adamancy with which certain scholars caution the unwary researcher against using stepwise methods, their use continues unabated. The problems associated with stepwise methods, i.e., incorrect degrees of freedom calculated by computer packages, sampling error capitalization, and failure to select the best subset of predictors, are significant. Some of the alternatives to address these problems, such as manually correcting degrees of freedom, cross-validation procedures, and all-possible subsets analyses, have been forwarded in the present paper. Perhaps the best alternative for researchers is to remember that computer packages do what they are programmed to do, and do not provide interpretation of results. It is the researcher who must design the study and choose the statistical procedures that will allow him or her the greatest opportunity for making sense of the results on the computer printouts.