People are more accurate at recognizing faces of individuals of their own race than individuals of another race; this well-replicated phenomenon is known as the cross-race effect (CRE). The reasons underlying the CRE are not clear; research indicates that expertise with own- versus cross-race faces may play a role, but there is also evidence that social categorization is a factor. Two experiments examine the roles of expertise and categorization by comparing the effects of cross-race faces to novel stimuli (Experiment 1) and rival university members (Experiment 2). Experiment 1 used inversion as a test of expertise and holistic processing to compare White faces, Black faces, and Greebles. In addition, the EZ Diffusion model was utilized to gather insight into the components underlying the differences in accuracy and reaction time. Results indicate that, for most measures, Black faces shared more similarities with Greebles than White faces, pointing to a strong component of expertise in the CRE. In Experiment 2, upright and inverted images of White and Black males were identified as members of the participants' own or rival university. Again, the EZ Diffusion model was applied to the results. A cross-race effect was found, but contrary to work by other researchers, no cross-university effect was detected, again indicating that expertise, rather than categorization, is a critical factor in the CRE. The results are discussed within the context of models that incorporate both perceptual expertise and social categorization, as well as how factors such as saliency might have affected the results.