SR4-Fit: Interpretable Rule-Based Learning Framework for Informative and Trustworthy Decision-Making

Abstract

In many high-stakes applications, machine learning is still dominated by blackbox models that require post hoc explanations to justify their predictions. These explanations are often unreliable, since they do not reflect what the model is computing, which limits accountability and trust. A natural alternative is to use models that are interpretable by design. However, existing rule-based approaches, such as RuleFit and decision trees, while transparent, often lack stability and predictive strength, reinforcing the perception of a trade-off between accuracy and interpretability. To address this, we propose Sparse Relaxed Regularized Regression Rule-Fit (SR4-Fit), an algorithm for both classification and regression that produces compact and stable rule sets without sacrificing performance. Using demographic data from the U.S. Census Bureau’s American Community Survey, SR4-Fit predicts U.S. House election outcomes with high accuracy and interpretability while uncovering demographic interactions missed by black-box models. We further validate SR4-Fit across multiple commonly used machine learning datasets, six in classification and eight in regression, where it consistently demonstrates strong performance in terms of accuracy, stability, and compactness. Across experiments, the SR4-Fit was comparable in terms of accuracy to black box models and surpassed them in terms of robustness and compactness of the model. The algorithm surpassed existing interpretable rule-based algorithms such as RuleFit and decision trees in terms of accuracy and robustness and was comparable in terms of compactness, thereby generating stable and interpretable rule sets while maintaining better predictive performance, thus addressing the traditional trade-off between model interpretability and predictive capability.