Warning Effectiveness: What Do We Need to Know

The forensic area of practice for human factors engineers has brought into sharp focus the differences of opinion which exist regarding the effectiveness of warnings in bringing about safe behavior on the part of the user of a product. This paper addresses the major issues which the authors believe must be researched further to provide the definitive answers needed regarding the effectiveness of warnings in a variety of possible applications. A review of the literature will demonstrate that there are few studies of warning effectiveness per se, while there are many studies that address such issues as the need for warnings and presumed criteria for preparing effective warnings. It is suggested that further research is needed which addresses warning effectiveness in actual use situations, and in turn identifies the importance of such variables as stimulus energy level, information overload, risk perception, cost of compliance and the interaction of warnings, instructions and training.

"grks paper will highlight t h e need for more studies to identify the significant variables related to warning effectiveness from the perspectives of an expert in tort litigation and research in an academic ceemunity. There are other applied research issues of warning effectiveness Erom the perspective of the designer/manufacturer of a product which are beyand the scope of this paper which are also imporlant in the overall question of warnings effectiveness.

Background
It has been 10 years since Dorris and Purswell (1977) published one of the first studies where warning effectiveness was empirically evaluated. In a second paper, (1978) a list of needed research was presented, including the following issues: 1. Optimum amount of information to be 2. Symbolic versus verbal warning 3 . Need for an appropriate methodology 4. Need to understand the factors that presented effectiveness for studying behavior influence responses to warnings.
The recent two-volume work of Miller and Lehto (1986) provides an updated discussion of these and other research issues which confront one when the question of warning effectiveness is considered.
However, of the 388 reference sources in their annotated bibliography on the topic of warnings, only 10 sources are listed as related to an analysis of warnings effectiveness, and o f these, only 6 actually cite any experimental results. The literature is therefore still not very complete in identifying the variables which are related to the design of effective warnings, and the information which does exist is not very enlightening regarding the design of effective warnings because most of the warnings investigated were not effective in bringing about safe behavior.
The paucity of actual studies of warning effectiveness in the literature is perhaps not too surprising.
These studies are difficult to perform for many products due to the logistics of collecting experimental observations in general, and the problem of inadvertantly introducing an experimental bias, i.e., the subject is influenced to read the warning and/or behave safely if he perceives that the purpose of the study is to evaluate warning effectiveness.
Need for a Conceptual Model ---While it is possible to suggest goals to be achieved in warnings design, and to provide some guidance for designers (Dorris and Purswell, 1978;Peters, 1984: Rosenberg, 1981Cunitz, 1981) it is now becoming clear that additional guidance based on sound research data is needed if more effective warnings are to be designed. In order to develop this data, our conceptual model of the warnings process must be refined to highlight the variables to be studied. Miller and Lehto (1986) have suggested a conceptual model based on the following sequential steps for a warning to be effective: 1.
The subject must be exposed to the warning stimulus The subject must attend to the stimulus There must be active processing of the warning message The subject must comprehend and agree with the warning message The subject may be required to store the warning message as well perform search and retrieval functions. An appropriate response must be selected The subject must perform the response The response must be adequate to avoid the injury Note that it is possible to have a reasonably high success rate in the population for each of these sequential steps and still have a low overall effectiveness rate for the warning. The next sections of this paper will present some of the important research needs as related to one or more of these steps in the warning process.

Stimulus: Energy Level and Contact
Many human factors/ergonomics texts (McCormick, 1976: for example) present guidelines for designing the warning stimulus so that it has sufficient energy to be perceived under a given level of illumination at some distance. Similarly, information is presented regarding the location of the warning in the visual field for optimum contact. If a person is inclined to seek warning information, then the stimulus energy levels specified in these texts for various conditions of use (are probably adequate for warnings design. However, there are many possible variables which can intervene to prevent warning information from being sensed. Miller and Lehto (1986) have used the term "filtering" to describe the effects of some of these variables on the sensing process.
The net effect of these filtering variables is that most users of a product will not read the warning information. There is a definite need to perform research to better understand the following filtering variables: 1,Information overload can occur in at least four ways : a. Warning lists frequently include numerous items, thus raising questions about a user's ability to perceive/recall items in the middle of the list.
b. The contents of i3 single warning may to too extensive, resulting in the percept ion that the warning is :Less effective (Wogalter et al.,, 1985).
c. There may be too many individual warnings placed in the field of view.
d. There may be other non-warning stimuli which are given priority over the warning stimuli.
The need exists for more research to understand the points at which information overload is likely to occur for each of the types of overload listed above.
A related research problem is to determine a satisfactory methodology for prioritization of individual warnings where space is limited and there are more warnings than can be accomodated in the space available.

2.
Faulty risk assessment can result in an individual failing to look for a warning or to ignore a warning if one is perceived to be present. Dorris and Tabrizi (1978), Slovic, et al. (19801, Godfrey et al. (1983) have all noted the problems which exist in persons performing adequate risk assessments for products.
More recently, Purswell, et a1.(1986a) found that the amount of risk information provided did not significantly influence the subjective rating of hazard perceived.
There is thus a need for more research to develop information concerning the way individuals use perceived risk information. In a sense, it is a "chicken and egg" type problem, i.e., a warning will not be read because a hazard is not perceived, and a hazard cannot be perceived without the warning information.
3. is a concept that has been suggested by Benign experience versus 5 warning Karnes et al. (1986) as one reason that warnings are filtered.
The hypothesis is that if one is regularly exposed to a warning about a hazard while at the same time exposing themselves to the hazard the warning addresses without being injured, the warning will. likely be filtered and even changes to improve the conspicuity of the warning will not likely prove to be effective.

Comprehension of Warnings
There are at least four types of problems in the area of warning comprehension where more research is needed: 1. The meaningfulness of "signal" words in calling attention to the severity of injury which can result if the warning is ignored. While it has been suggested by the FMC labeling system that the words "danger", "warning" and "caution" should be used to represent a hierarchy of decreasing potential for harm in designing warnings, this suggested format is not clearly supported by the research reported to date (Bresnahan and Bryk, 1975;Nikmorad, 1985). Miller and Lehto (1986) suggest that a better system might be extreme-danger, serious-danger and moderate-danger.

2.
The reading courprtshension level r e s r e d to understand the warning. If -messages are evaluated using one of the several tests developed for readability, it will be found that there is -a wide range of values, starting with grade four or five and continuing through grade twelve or higher. A related problem is the design of warning messages where a chemical compound is involved.
It is usually necessary to include a realtively large number of words to explain the health and safety consequences of exposure, the proper protective measures, and first aid measures when exposure does occur. In each case, there appears to be a definite tradeoff between the use of a smaller number of words with a more exact meaning versus a larger number of smaller words with less exact meanings.
3. The meaningfulness of symbol in l i e u o f words to communicate warning information.
Ideally, warnings should be designed to be language independent where there is a significant probability that the intended recipient does not read English, or whatever language is used in preparing the warning. There is a need for more research information concerning such things as the culturally derived meaning of various symbols, and the correct approach for designing pictorials to communicate different concepts.
4. -The meaningfulness of warnings as 5 function of the task being performed7or stated another way, the effectiveness of warnings when presented in the context of instructions as compared to presenting the warnings in a separate list wit;mu"i t r t e conrext of the task being performed.
It is pcssible to find warnings that are presented in the form of a list at the beginning of the instructions versus being presented at the point where the warning is needed in the context of performing some operation with a product.
Some proponents of placing a warning list at the front of the instructions argue that the user may not notice the warning if it is imbedded in the instructions, while others argue that the warning will not be meaningful unless it is placed near the instructions to which it is related.
One of the authors has a research project underway at the present time to provide some insight into this area of warnings design.
Warningsand Memory deal with warnings design, it is a common experience to find that few warnings are stored in long term memory if they are more than six or seven lines long, or address more than this number of hazards in using a For persons who product. Surveys done by the authors of long term recall of the warnings on such products as spray paint cans, drain cleaners, electric power tools, or complex equipment such as cranes have demonstrated that most people cannot recall more than four or five items of information from the warnings.
There is, of course, a distinction between having safety knowledge and remembering that the source of that knowledge was originally a warning. There is a need to have more research done to better understand the reasons for this lack of long term recall, i.e., is it related to the general problem of lack of active processing of warning information as described by Miller and Lehto (1986). Wright (1980) has described this problem for a warning displayed next to an antacid product, noting that less than 10% of the purchasers of the product could remember even a portion of the warning when asked upon leaving the store.
Given that most warnings are not generally committed to long term memory, then there arises the question of short term memory or' w a r i i i i i y s , and in particular the question of how our short term memory behaves in the presence of different stressors or distractors which may be present in the job task environment. More information is needed regarding the best process for committing warning information to long term memory and then recalling it when it is needed. Of course, it is possible that most people do not remember warnings because they filter the information and it never acutally reaches long term memory (McCormick, 1976).

The Decision Making Process
-Of all the areas where more research is needed, not just in the area of warnings, but in other areas of safety as well, the area of decision making seems to be among the most important. Slovic (1977Slovic ( , 1978 notes that people do not use risk information very objectively and consider that accidents are rare events which will not happen to them.
The authors have noted this phenomenon often in performing safety research.
It might be hypothesized that there is a threshold of perceived probability of an injury which must be reached before a person's behavior will be influenced by risk information such as contained in a warning.
This perceived probability threshold may be relatively high, i.e., l/l00 before most persons will respond to knowledge about hazards as presented in a warning.
Fortunately, most products do not have this high a probability of injury per use, and therefore warning messages are either not perceived or not heeded. It is very important to obtain a better understanding of how our risk pereeption/riSk acceptance interacts with the warning process. Godfrey et al. (1985) noted that there is another factor operating in our decision process when we are confronted with a warning.
The tenn "cost of compliance" was used to suggest why the warnings may not be heeded.
If an individual complies with a warning, some cost in terms of money, timle, effort or perceived enjoyment is extracted from the individual, and the perceived benefit must outweigh t'hese costs. There may also be a benefit of noncompliance that is of significant importance to the individual. It may take the form of maintaining a macho image, more money for increased production, etc. Perhaps the costs and benefits o f both compliance and noncompliance play a signific'ant role in warning effectiveness. Since there is frequently a difference in the dimensions of cost and benefit, it appears, implicitly at least, that the individual constructs some type of utility function for making the tradeoff. B'ecause there are likely many such transient functions employed by an individual, it may be difficult to describe such functions in a manner that renders them useful to perform safety analyses. The most promising descriptor found to date for such behavior is a questionnaire developed by Purswell, et al., (1986b) to measure attitudes about risk taking. Further research using this concept should add to our understanding of why a person behaves safely or not when presented with a warning.

RESEARCH STRATEGY
A s noted in the beginning of this paper, it is difficult to perform research in the area of warnings design, and yet it is very important to complete some of the research suggested if we are to avoid the present controversies among human factors engineers and reduce the legal rhetoric which exists today in lieu of sound information. Some possible approaches are as follows:

1.
To the extent possible, future accident investigation efforts should focus on such issues as hazard aware-1119 ness, perception of risk, and accident avoidance information which was obtained from a warning as compared to other sources.

2.
Experimental approaches utilizing discriminant analysis should be used to better understand the relative influence of a variety of personal, product (including warnings) and environmental factors on safe behavior.
3. Carefully controlled studies should be done to determine the relative influence of human and environmental variables on the major factors discussed in this paper, and in turn the relative influence and importance of those factors on the success of the overall warnings/instruction/training process.