The relentless pursuit of construct validity: Why employment interviews predict job performance?

Abstract

Employment interviews are one of the most frequently researched areas in the human resources management research (Buckley, Norris, & Wiese, 2000). Over the last eighty years, the employment interview research has gone through several phases. Fifty years ago, researchers seemed to have little hope for the criterion-related validity of employment interviews (Ulrich & Trumbo, 1965; Wagner, 1949). However, in the 90s several methodological advancements furnished evidence to support the criterion validity of employment interviews (Huffcutt & Arthur, 1994; McDaniel, Whetzel, Schmidt, & Maurer, 1994). Structuring interviews has been the major antecedent of increasing validity of interviews since then. The patterned behavior description interviews (Janz, 1982) and the situational interviews (Latham, Saari, Pursell, & Campion, 1980) have become the two most widely used techniques of conducting structured employment interviews. With all these advancements, any doubts regarding the criterion validity of structured employment interviews have been nullified. This development has freed researchers to turn their attention toward a newer set of issues in this line of research.

One new question attracting considerable attention from researchers is "why do employment interviews predict performance?" On the surface it seems to be a straightforward question with an easy answer. However, research so far has provided equivocal results regarding the construct validity of employment interviews. My three-essay dissertation focuses on resolving those issues that plague the construct validity of employment interviews. In the first essay, I review the existing research that is concerned with the constructs underlying employment interviews. I examine various issues that emerge during the stages of interview design, administration, assessment, and analysis. I pay special attention to the theoretical issues that hinder construct validity evidence. More specifically, I highlight how grounding employment interviews in the theory of job performance (Campbell, McCloy, Oppler, & Sager, 1993) might positively influence construct validation efforts. I also stress the need to view an employment interview as a predictor method (rather than as a predictor construct) and emphasize the unique construct validation challenges that any predictor method (e.g., assessment center) faces. The purpose of the first study is to lay out a comprehensive framework that guides future research.

Many researchers have shown that different types of interview questions (e.g., behavioral questions, situational questions, and general questions) tap different constructs (e.g., Conway & Peneno, 1999). My second essay explores the construct validity of past behavior description interviews (PBDI). To date several studies have explored the constructs underlying PBDI questions. Social skills, experience, motivation, and intelligence have been proposed as some of the constructs that explain why PBDI questions predict performance (O'Leary, 2004). However, there seems to be little consensus on the specific constructs that are tapped by the different interview types. PBDIs are designed on the premise that a

pattern of past behaviors is a best predictor of future behaviors. The structuring efforts are aimed at ensuring that interview responses only assess the quality of past behaviors, and that no other biases (e.g., likeability of a candidate, personal biases of an interviewer, and impression management by a candidate) influence the evaluation of a candidate. If PBDI questions only measure the quality of past behaviors then the assessment of candidates based on oral interviews and the assessment based on written responses should be the same. I challenge this assertion and argue that an interview is primarily a social interaction process. Thus, even highly structured interviews, such as PBDI, measure additional constructs such as extraversion, emotional stability, self-esteem, and self-efficacy.

The third study explores the construct validity and incremental validity of general interview questions. Despite research evidence that supports the use of PBDI- and SI-type questions, managers continue using general questions in actual job interviews. In this study, I propose how to make the general interview questions more effective. I propose that researchers can increase the validity of general questions when they have an a priori understanding of underlying constructs, they ground the interview design in a relevant theory, and they make the intended dimensions transparent to the candidates. I argue that general type of questions can assess certain constructs such as values and personality, and thus add incremental validity to the interview process. In addition, by incorporating general questions in the interview design, interviews achieve the flexibility desired by practitioners and increase their comfort level with the interview process.