Performance evaluation of a Cadre Forensics TopMatch-GS 3D system for cartridge case comparisons

Abstract

This research presented a series of commonly used firearms and commonly used ammunition brands to evaluate the Cadre Forensics TopMatch-GS 3D system overall and to test for differences between same ammunition and different ammunition comparison scores. For this study, ten casework firearms were shot, producing 30 cartridge cases from each, except two firearms that only had 20 cartridge cases fired from each; for a total of 280 cartridge cases. The ammunition manufacturers used were Blazer, CCI, MaxxTech, Wolf, Remington, and Winchester. This study utilized match and nonmatch scatterplots and ROC curves to evaluate the overall performance of the Cadre Forensics TopMatch-GS 3D. All cartridge cases were inter-compared using TopMatch breech face impression and aperture sheer algorithms for 39,060 comparisons. This research aimed to validate the Cadre Forensics TopMatch-GS 3D, evaluate automated algorithm performance on various casework firearms, and examine possible differences in automated scores between ammunition manufacturers on system performance. The tested firearms and ammunition brands performed well using the Cadre Forensics algorithms. The AUC for all firearms in this study was 0.956 indicating the automated breech face algorithm can differentiate between same and different sources. The same source and same ammunition AUC was 0.988 while the same source and different ammunition comparisons performed slightly worse with an AUC of 0.952. A procedure for comparing AUCs based on the Mann-Whitney test was used to test for significance and the resulting p-value was less than the 0.05 threshold (or 95% level of confidence) indicating that same ammunition brands performed significantly better than different ammunition brands. MaxxTech and Wolf were brass primed and scored exceptionally well regardless of the firearm. Winchester and Remington resulted in lower scores when compared to other ammunition brands. Aperture shears under 800 microns in length provided misleading aperture shear results. Running the breech face algorithm over the combined score feature is recommended as the combined score defaults to the breech face score if the system believes the aperture shear values may be misleading. The data collected in this research will allow for further development and improvement to the Cadre Forensics comparison algorithm and open the door for future research opportunities using this system.