TRACER: A Reliability-First GemNet Baseline for Trustworthy Computational Materials Discovery

Abstract

Computational materials discovery is increasingly driven by graph neural networks (GNNs); however, deployment is constrained by uncertain robustness and inadequately validated uncertainty quantification (UQ). The ALIGNN, MACE, and Open Catalyst models have made strides, but there is still no standardized framework for reproducibility, fairness, and dependability. We introduce Transparent and Reliable Accuracy, Confidence, and Error Ranking (TRACER), a comprehensive, transparent, and repeatable reliability-first pipeline built on a GemNet-based Graph Neural Network (GNN). Using a held-out test set investigate robustness via sensitivity to graph cutoff, architectural depth, and training-data fraction. Achieving competitive accuracy, single GemNet model records a Mean Absolute Error (MAE) of 0.0370 eVatom−1 on JARVIS-DFT, representing a 25.8% reduction in MAE compared to an identical split ALIGNN baseline. UQ benchmarking demonstrates that deep ensembles provide informative uncertainty estimates that strongly correlate with error, proving superior for “hard-case” identification and triage. We also present valuable negative results: FiLM-based domain adaptation provides no significant benefit on this single domain task, and a material-aware “Gate-Hard” heuristic is outperformed by simple variance-only ranking for identifying high-error cases. The framework offers substantial operational utility, demonstrating F1 score of 0.378 (N=3 ensemble, 20% budget) compared to 0.325 for random selection in capturing high error cases compared to random selection when working with tight computational budgets. Confirmed by generalization to Matbench Perovskites, TRACER provides reproducible and confidence-aware computational materials discovery, as well as a strong predictor.