Machine Learning Model Introspection: Framing

I'm motivated by a perceived lack of study and effort in the machine learning community to care about visualization and use it effectively. If I can score in the top of a data science competition while only using pre-packaged models and summary statistics, why do I should I care about visualization? Well, if we can call machine learning 'automated pattern recognition', then we ought to see the patterns recognized. We want to understand if the model is arriving at generalizable predictions, and be able present those patterns to others. Even for single-minded chasing of performance metrics, we can use visualization and model introspection to tighten the feedback loop from the feature engineering, parameter tuning, and model structure to quality of predictions.

A machine learning model should contain information about a generalizable pattern that is recognized to produce some output. Visualization should represent these patterns. Ideally, viewers are able to map from points in the input space to predictions generated. However, models are often too complex for this. Even so, an effective visualization or model introspection should show the most important feature correlations, strengths, and feature interactions. We'd like to know where the model does well on some test set, and where it fails. We may even want to know how it extracted the patterns that it did. Most importantly, the visualization should be useful for improving the model or evaluating the design of the machine learning approach.

So we can come up with a few criteria:

• What the model predicts: mapping from the input space to the output space
• How it predicts: the rules and structure within the model that generate the prediction
• Why it predicts: the relation between the training data and the extracted patterns
• Where it fails: how the performance of the prediction varies within the state space

A series of visualizations that addresses each of these criteria can be sufficient, and may even be simpler, but integration excels when possible.

Though we may think that we should just pass off the n-dimensional pattern recognition problem onto our machine learning models, we can still fight the good fight of compressing high-dimensional information onto our ~two dimensional projections. Our own faculties still work.

In sum, I'm looking for opportunities for mutual augmentation of human and automated pattern recognition.