FLAIRS 2019 Tutorial: How to Determine the Optimal Anomaly Detection Method For Your Application

We introduce a framework for automating the classification of time series and choice of anomaly detection method. The early detection of anomalies is vital for ensuring undisrupted business and efficient troubleshooting. Unfortunately, there is no one best way to detect all anomalies across a variety of domains; such a methodology is a myth given that time series can display a wide range of behaviors. In addition, what behavior is anomalous can differ from application to application. In this tutorial we introduce a framework that helps you determine the best anomaly detection method for your application based on the characteristics the time series possesses (e.g. seasonality, concept drift, etc.). Participants will get hands-on experience applying various anomaly detection methods to several example datasets. We will then discuss the results they observe and how best to evaluate them.

Requirements

This tutorial is hands-on and participants will be able to run the code and experiment with the datasets. In order to participate, a laptop with a current version of Windows, macOS, or Linux will be required. Participants without laptops will still be able to learn the concepts and methods from the presentation, but will miss out on the hands-on experience.

VirtualBox will be used for the exercises portion of the tutorial. Please install VirtualBox to run the virtual machine provided. It has also been tested to work directly in VMWare Fusion 8+, and may be imported into other virtualization software such as Hyper-V, but this is beyond the scope of the tutorial. You will need about 10GB of hard disk space free for VirtualBox and the VM image.

Materials

Slides from talk

VirtualBox installers and the following VM image will distributed on USB drives during the tutorial as well to prevent network bottlenecks.

  1. Download the following:

  2. Open VirtualBox and click Add

  3. Select the FLAIRS2019Tutorial.vbox config file from where you saved it

  4. Start the VM, the Jupyter notebook server and Firefox will automatically start up in the VM. For each exercise, navigate to the corresponding folder and click on the ipynb file inside to launch the exercise notebook.

Agenda

During the introduction, attendees will be installing supplied VM image containing Jupyter notebooks and datasets to follow along. Help will be provided as needed to make sure everyone is able to use the materials.

Introduction

(15 min) Who are the speakers? What is an anomaly and why is its detection important? What has anomaly detection been used in? Why is anomaly detection difficult?

Time Series Characteristics and How to Detect Them

(70 min) Define time series characteristics and how to detect them:

Dataset Resources

(5min) Go over valuable resources for time series datasets

Break (30 min)
Anomaly Detection Methods

(80 min) Introduce and briefly discuss the following anomaly detection methods:

Evaluation Strategies

(5 min) Define several evaluation strategies:

Results and Conclusion

(5 min) Give guidelines on what anomaly detection methods work better on certain characteristics than others.

Open Question and Discussion (time permitting)

Organizers

Cynthia Freeman1 - Research Engineer Verint Intelligent Self-Service

Bio: Cynthia Freeman is a Research Engineer at Verint, a developer of conversational AI systems. She is currently pursuing her PhD in Computer Science at the University of New Mexico, where she works on time series analysis and developing new anomaly detection methods. She holds an MS in Applied Mathematics from the University of Washington and a BS in Mathematics from Gonzaga University.

Ian Beaver, PhD2 - Lead Research Engineer Verint Intelligent Self-Service

Bio: Ian Beaver has been publishing discoveries in the field of AI since 2005 on topics surrounding human-computer interactions such as gesture recognition, user preference learning, and communication with multi-modal automated assistants. Ian has presented his work at various academic and industry conferences and authored numerous patents within the field of human language technology. Ian is currently focused on dialog systems and means to detect and resolve misunderstandings between humans and machines. He is also leading a team in finding ways to optimize human productivity by way of automation and augmentation, using symbiotic relationships with machines. Ian received his PhD in Computer Science from the University of New Mexico. He also holds a BS and MS in Computer Science from Eastern Washington University.


  1. cynthia.freeman@verint.com 

  2. ian.beaver@verint.com