Novel Algorithms for Interpretable and Semi-Supervised Machine Learning

Abstract

Pseudo-labeling (PL) has historically been considered synonymous with model-provided weak supervision in the context of semi-supervised learning (SSL). In that context, pseudo-labels are simply an inferior version of human supervision. Framing pseudo-labeling this way provides an overly restrictive view of how categorical model outputs can and have been used in prior work. In this dissertation I argue that pseudo-labels can be effectively applied in ways other than as weak supervision. Recognizing the shared use of pseudo-labels across algorithms not historically grouped together yields new insights into how we can use them. I present a formal definition of pseudo-labels and a taxonomy of pseudo-labeling algorithms. Under this definition the supervisory role of the pseudo-label is not essential; the important aspect of pseudo-labels is that they are the product of a model. That model can be optimized to encourage desired properties of those pseudo-labels such as their utility to the learning process or explainability. My definition allows the conceptualization of new PL algorithms, three of which are presented here. Meta co-training (MCT) is a novel co-training-style PL algorithm for SSL that trains two collaborating models. Each model optimizes its pseudo-labels to reduce the loss of the other model. Policy gradient pseudo-labeling (PGPL) draws connections between prior PL works and reinforcement learning (RL) methods; these connections lead to an RL-inspired method that learns to maximize the value of pseudo-labels during training. Localized additive explanations (LAX) applies PL to the problem of explainable artificial intelligence, training a model to jointly explain and predict. My contributions support the central claim that PL is a versatile tool that can be applied to machine learning algorithms both in SSL and outside of SSL.