Model Monitoring: Detecting and Analyzing Data Issues

Machine learning models are deployed to production environments increasingly more often. Unlike traditional coded applications, serving ML models are fully data-driven. Unexpected input data, which the model has never seen in training and is not prepared to handle, may produce garbage model output that will be treated as valid by the rest of the application without producing any error or exception. For this reason production models introduce a new monitoring requirement, a model monitoring, that is not addressed by the existing tools. This talk focuses on monitoring and detection of production data issues, such as data anomalies and drift.