Understanding classifier errors by examining influential neighbors.

Our Research / Publications

janelia7_blocks-janelia7_biblio_header | block

IEEE Conference on Computer Vision and Pattern Recognition. 2015 Jul

Understanding classifier errors by examining influential neighbors. Branson Lab

Kabra Mayank, Robie Alice, Branson Kristin

Full Text (PDF)

Supplementary Information (PDF)

Extended Abstract (PDF)

janelia7_blocks-janelia7_biblio_abstract | block

Abstract

Modern supervised learning algorithms can learn very accurate and complex discriminating functions. But when these classifiers fail, this complexity can also be a drawback because there is no easy, intuitive way to diagnose why they are failing and remedy the problem. This important question has received little attention. To address this problem, we propose a novel method to analyze and understand a classifier's errors. Our method centers around a measure of how much influence a training example has on the classifier's prediction for a test example. To understand why a classifier is mispredicting the label of a given test example, the user can find and review the most influential training examples that caused this misprediction, allowing them to focus their attention on relevant areas of the data space. This will aid the user in determining if and how the training data is inconsistently labeled or lacking in diversity, or if the feature representation is insufficient. As computing the influence of each training example is computationally impractical, we propose a novel distance metric to approximate influence for boosting classifiers that is fast enough to be used interactively. We also show several novel use paradigms of our distance metric. Through experiments, we show that it can be used to find incorrectly or inconsistently labeled training examples, to find specific areas of the data space that need more training data, and to gain insight into which features are missing from the current representation.

Code is available at https://github.com/kristinbranson/InfluentialNeighbors.

node:body | entity_field

janelia7_blocks-janelia7_biblio_authors | block

janelia7_blocks-janelia7_biblio_tools | block

Main Menu (Mobile)- Block

Main Menu - Block

Understanding classifier errors by examining influential neighbors.

Abstract

Janelia Authors