Supervised learning models are one of the most fundamental classes of machine learning models. Viewing supervised learning from a probabilistic perspective, the set of training data to which the model is fitted is usually assumed to follow a stationary distribution. However, this stationarity assumption is often violated in a phenomenon called concept drift, which refers to changes over time in the predictive relationship between covariates X and a response variable Y and can render trained models suboptimal or obsolete. We develop a comprehensive and computationally efficient framework for detecting, monitoring, and diagnosing concept drift. Specifically, we monitor the Fisher score vector, defined as the gradient of the log-likelihood for the fitted model, using a form of multivariate exponentially weighted moving average, which monitors for general changes in the mean of a random vector. Advantages of the proposed score-based framework include applicability to broad classes of parametric models, more powerful detection of changes, and inherent diagnostic capabilities for helping to identify the nature of the changes.
Anh T. Bui received a B.S. degree in electrical engineering from Hanoi University of Science and Technology, Vietnam, the M.S. degree in industrial and management engineering from Pohang University of Science and Technology, South Korea, and the Ph.D. degree in industrial engineering & management sciences from Northwestern University, USA. He is currently an assistant professor in the Department of Statistical Sciences & Operations Research at Virginia Commonwealth University, USA. His research interests lie in statistics and machine learning, for analyzing data from manufacturing, healthcare, and other enterprise systems. Dr. Bui is a recipient of the Lloyd S. Nelson Award and the Frank Wilcoxon Award from the American Society for Quality.