Anomaly Detection
What makes anomaly detection hard is that examples of the anomaly are almost never available. This topic covers methods that catch a change in distribution from normal data alone, built on density-ratio estimation. The main result is Non-negative Bregman Divergence Minimization for Deep Direct Density Ratio Estimation (Kato and Teshima, 2021), published at ICML 2021.
Problems addressed
Fraud, equipment failure, and degradation in data quality all have to be caught without a large body of labeled anomalies, which in practice cannot be assembled. So the formulation takes the distribution of normal data as the reference and measures, through a density ratio, how far a new observation sits from it. Estimating the ratio with a deep neural network buys expressive power, but it overfits in a particular way: the training loss alone falls to an implausibly small value. We analyzed that behavior and stabilized estimation with deep models by adding a non-negativity correction to the empirical loss.
On the applied side, we are formulating audit sampling for financial statements as a sequential hypothesis test and attaching statistical guarantees to it. Anomaly detection is not the best answer to every problem. Where anomalies are available in quantity, ordinary classification is the more direct route, and these methods earn their place where such examples are hard to come by.
Core research
The Non-negative Bregman Density Ratio Estimation page explains how the overfitting arises, how the correction is designed, and how it performs on anomaly detection.
Research thread
- 2026
- Sequential Audit Sampling with Statistical Guarantees. Designs audit sampling as a sequential test and gives it guarantees.
- 2023
- Unified Perspective on Probability Divergence via Maximum Likelihood Density Ratio Estimation (AISTATS 2023). A theoretical study that unifies the measures of difference between distributions from the standpoint of density-ratio estimation.
Relation to earlier work
The body of methods that use density ratios for anomaly detection and distribution shift was systematized by Masashi Sugiyama and coauthors (Sugiyama, Suzuki, and Kanamori, 2012, Density Ratio Estimation in Machine Learning). Our work pins down what goes wrong when those methods meet deep models, and supplies a correction. The Density-Ratio Estimation page covers the theoretical placement in detail.
Where it applies
The settings we have in mind are fraud detection, the monitoring of equipment and transactions, and data quality management, where a detection system has to be built with few examples of the anomaly in hand. Bringing statistical methods into audit work is also covered. For inquiries of this kind, write to mkato-csecon@g.ecc.u-tokyo.ac.jp.