Response Surface Methodology and Optimization — A Possible Pitfall

Recently Response Surface Methodology (RSM) has come to the attention of researchers in the area of human performance, as evidenced by a special issue of Human Factors (August 1973) devoted to the topic. These articles, as well as others in the area, neglect to discuss the fact that RSM can be used as an optimum seeking technique. There can be, however, a serious practical problem or question that may arise in using the optimization technique associated with RSM (Cochran 1973). Those coefficients of the full model found to be insignificant can be eliminated and a new, or ~bbreviated, model determined with only the significant coefficients retained. The optimization procedure may now be applied, using either the full or the abbreviated model with the possibility of completely different outcomes. Considerable differences between the values of the independent and dependent variables may be found at the optimum points between the full and various abbreviated models.


INTRODUCTION
Response surface methodology (RSM), a regression technique developed by Box and Wilson (1951), requires a minimum number of data points to obtain a full second order model. It is very efficient and therefore useful in a variety of situations for estimating the parameters (regression coefficients) of a second order model. Recently RSM has come to the attention of researchers in the area of human performance, as evidenced by a special issue of Human Factors  devoted to the topic. Within this issue Clark and Williges (1973) discuss and demonstrate the uses of RSM in experiments on human performance. This article, as well as others in the area, neglects to discuss the fact that RSM can be used as an optimum seeking technique.
Through a series of experiments, the experimenter using RSM is able to approach and finally describe the response surface about the optimal point (if one exists). There can be, however, a serious practical problem or question that may arise in using the optimization technique associated with RSM (Cochran 1973). Contradictory results may be obtained depending on the selection of a. As is explained in detail in Myers (1971), in the use of RSM as an optimum seeking technique, 441 it is customary to conduct a series of experimentseach describing an area closer to the optimum performance point than the one before. From each experiment the coefficients of the second order regression model are estimated. The regression model is then used to find the stationary point (maximum, minimum, or saddlepoint). The stationary point is found by taking the partial derivatives (with respect to the regression coefficients) setting the resulting derivatives equal to zero, and solving the resulting equations simultaneously. The solution must be verified to be a maximum, minimum or saddle point by determining the signs of the eigen values. If the eigen values are (1) all positive -the point is a minimum point; (2) all negative -the point is a maximum; or (3) mixed -the point is a saddle point. The next experiment is then set up such that the experimental points encompass the stationary point of the regression model of the previous experiment. This process is repeated until the stationary point (hopefully an optimum point) is at or near the center of the model's experimental points.
PROBLEM l{hen some of the coefficients of the regres7 sion model are not significant, a problem may arise in the use of the optimization technique discussed above. The significance of all coefficients can be evaluated, using analysis of variance, noting that significance or lack of it is highly dependent upon the value of a specified by the experimenter. Those coefficients found to be insignificant can be eliminated and a new, or abbreviated, model determined with only the significant coefficients retained. The optimization procedure may now be applied, using either the complete or the abbreviated model with the possibility of completely different outcomes.
There is a dilemma here that concerns whether the experimenter uses the full or abbreviated model. This is made worse when it is realized that the selection of a can have an effect on the composition of the abbreviated model. In fact, there could be numerous abbreviated models which depend only upon the experimenter's selection of a.
The existence of alternative models forces the experimenter to make a decision as to which model to use. One thing supporting the use of the complete model is that it most accurately predicts the data from which it was developed. On the other hand, an abbreviated model may be used because it contains only those terms which have a significant effect on the predicted value.
The objective of this research was to examine several sets of data and demonstrate the possible consequences in optimizing on the different models -full or abbreviated. This should provide very useful information for those researchers using RSM as an optimizing technique.
Sets of data involving three factors (Table  1) from Clark et al (1971) (Yl and Y2), Myers (1971) (Y3), and Cochran and Cox (1964) (Y4) were examined and analyzed in the following manner: 1. Estimates of the regression coefficients for a full second order model were determined with an appropriate analysis of variance ANOVA) using the computer program of Clark et al (1971). 2. Significant terms (both a = .05 and a = .025) were determined from the ANOVA and new regression coefficients determined using only the significant terms. 3. The stationary point was determined for complete models, the abbreviated models using a = .05 and a = .025. 4. Differences between the stationary points of complete and abbreviated models were noted and the percent difference from the complete to the abbreviated models calculated.  In analyzing two sets of data from Cochran (1973) which used a RSM design in five variables, almost the same procedure was followed, the difference being that the full and only one abbreviated 442 model (a = .05) were used. Therefore, data from four, three factor, and two, five factor RSM designs were examined to ascertain and demonstrate changes in the stationary point due to the level of significance used in the model determination. Table 2 contains coefficients of the three factor coded models arrived at using RSM. "Yl" indicates the full second order RSM model for the first set of data, "Yl,el" indicates the second order model for the same data arrived at by including only those terms shown by ANOVA to be significant at that el. Upon examination of Table 2, it becomes apparent that deletion of insignificant terms can reduce the size of the model appreciably. The most pronounced case here is "Y3" vs "Y3,.025" where the number of terms is reduced from 9 to 2.

Three Factor Models
The coded independent variable values and the function value (Yi) at the stationary point are shown in Table 3 for the coded, full and abbreviated second order, three fa~tor models. In addition to the coded variable and function values, the percents of change from the full to the abbreviated models are given. The coded values of the independent variables in these examples change from the full to the abbreviated models by as little as 2.6% [X3 for the Y2 05 model] to as much as 368% [X2 for the Y4,.OS model]. Of the changes in the value of the independent variables (excluding those variables where their main effects and all interactions are eliminated from the model), 10 are equal to or greater than 100% and 13 are greater than 50%. Changes of these magnitutes and frequencies must have an effect on the function values (Yi) at their stationary points.
The relative changes in the uncoded independent variable values are of more importance than those of the coded independent variable values. Table 4 contains the uncoded independent variable values and the percent of change from the full models. The percents of change of the uncoded independent variables in these three examples range from 2.71 (X3 for the Y2,.05 model) to 39,326.47 (X2 for the Y4 .05 model) percent. If this extreme value is di~regarded, the range is still from 2.71 to 62.16 (Xl for the Y4 .05 model) percent (disregarding also those variabies eliminated from the model). The average percent of change from these models was 1982.14 percent with the extreme value included and 16.65 with it deleted. In any case, an average change of over 16 percent could be critical in using the experimental results to make decisions.
Although changes in the models from the full to the abbreviated ones cause changes in the values of the functions (Yi) at their respective stationary points, they do not appear to be as drastic as those for the variable values (coded or uncoded).    pendent variables (Xi'S) and the dependent variables (Yi's) may be surprising at first glance but is actually quite reasonable. This is due to the fact that only the most insignificant terms are eliminated. Therefore, these are the terms which least affect the regression line or surface. Because the dependent variable is affected by many terms, one term has to change considerably to change the dependent variable significantly. In addition, a large change in one independent variable may be negated by a change of another variable with the net result of little or no change in the value of the dependent variable.
stationary point for both full and abbreviated models. In addition, the percents of change in these values from one model to the next are included. Of the ten possible changes in the coded values of the independent variables, six are of 100% or greater and seven are of 50% or greater. Two variables were eliminated in the abbreviated models because they were not significant in their main effects or interaction terms. The changes in the coded values of the independent variables of the five factor models at the stationary point range from a low of 27.3% [X 4 for the YS,.05 model] to a high of 546.3% [X4 for the Y6,.OS model].

Five Factor Models
The results of the analyses of the five factor models are similar in most aspects to those for three factor models. Table 5 contains the coded five factor models arrived at using second order RSM. Consistent with the notation of the three factor models, the Ys indicates the full second and Ys .05 indicates the second order model for the same data arrived at by including only those terms shown by ANOVA to be significant at the 0.05 level. The reduction in the size of the model by elimination of insignificant terms is quite obvious and pronounced here. The reduction in the number of terms for Ys to Ys,.OS is from 21 to 6 and is from 21 to 9 for Y6 to Y6,.05' Table 6 shows the coded values of the five independent variables and related Yi at the Table 7 contains the uncoded values of the independent variables and related dependent variable values at the stationary points for the five factor models. The range of percents of change of the values of the uncoded independent variables was from 3.61 [Xl for the Y6, .05 model] to 523.00 [X4 for the Y6,.OS model] percent. The average percent change is 182.33 percent. It becomes obvious at this point that the selection of the a level can have drastic effects on the independent variable values at the stationary point.
The changes of the dependent variable values (Yi) at the stationary points were 79.6% (YS 05) to 870.2% (Y6,.OS) from the full to the abbre~ia ted models. These changes are more extreme than similar changes of the three factor models. This could be due to the fact that the reduction in the size of the full model by elimination of insigni-X, X 2   .. Example A look at the Y3 models will illustrate what happens to the independent and dependent variables as the different models are used. This example "involves an experiment from which the researcher attempts to gain an insight into the influence of sealing temperature (xl)' cooling bar temperature (x2), and % polyethylene additive (X3) on the seal strength in grams per inch of a breadwrapper stock." (Myers, 1971, page 78). All models in this case give stationary points which are maximums. The full model indicates the maximum of 10.6650 grams per inch of breadwrapper stock is attained when the sealing temperature (xl) is 234.45 degrees, the cooling bar temperature (x2) is 56.26 degrees, and the percent of polyethylene additive (x3) is 1.28%.
When the first abbreviated model (a = .05) is used, a maximum seal strength of 8.1079 grams per square inch is attained when the sealing temperature (xl) is 271.70 degrees, the cooling bar temperature (x2) is 55.00 degrees, and the percent of polyethylene is 0.79%. This model indicates the best seal strength is only 8.1079 rather than the 10.6650 attained for the full model. In addition, it indicates that the sealing temperature should be 37.25 degrees higher, cooling bar temperature should be 1.62 degrees lower and the percent polyethylene additive reduced from 1.28 to 0.79 (.49 percent less).
Using the second abbreviated model (a = 0.025) a maximum point of 8.8442 grams per inch is indicated when the percent polyethylene additive is 1.1 percent'and the other two variables are unspecified but within the limits of the experimental data.
The changes in the independent variable values and the dependent variable values in this case are considerable. In addition, they do not appear to be predictable or consistent. The maximum seal strength predicted goes from 10.6650, i.e., the full model, down to 8.1079 in the first abbreviated model (a = .05), and back up to 8.8442 in the second abbreviated model (a = .025). The value of the first independent variable increases from the full to the Y3,.05 model and then becomes undetermined in the Y3,.025 model. The value of the cooling bar temperature (unlike the sealing temperature) goes down slightly from the Y3 to the Y3,.05 model, and then it too becomes undetermined in the Y3,.025. In the progression from Y3 to Y3,.05 to Y3,.025, the optimum percent of polyethylene goes down from 1.28 to 0.79 percent, and then up to 1.1 percent. There does not appear to be any consistent pattern of change as one progresses from the full to the more abbreviated models.

CONCLUSIONS
This study has demonstrated a definite pitfall in the use of RSM, or any regression technique, as an optimizing technique. It was shown that the 445 stationary point (maximum, minimum, or saddle point) is' dependent upon how abbreviated a model is which is itself dependent upon the selection of a. Both the location (values of the independent variables) of the point and the value of the dependent variable may be altered considerably and unpredictably by the selection of a and therefore the model. This research has not, however, resolved the problem -it has only brought it to light. There is still a dilemma for which the experimenter is given no method of resolving. Examination of this problem for possible solution, such as determination of alternate criteria for determination of the correct model, should be a goal of future research in the area.