Distributed Anomaly Detection Systems for M2M Applications
The project aims to build a distributed anomaly detection system for an M2M environment. A toolbox containing a set of anomaly detection algorithms will be designed and tested for various anomalies under different M2M usage scenarios. To address the challenges of the M2M environment, we propose a distributed and hierarchical anomaly detection system. Under the proposed framework, there are a set of front-end lightweight detectors and a powerful back-end analysis center. The front-end detectors are responsible for collecting data for anomaly detection and computing approximation models based on computation resource constraints. This data is forwarded to the back-end analysis center to compute more complex correlations and models. We are interested in estimating the error bound between the approximation model and the exact model. That should tell us when and how we need to update our approximation model on the platform.
PI:Prof. Yuh-Jye Lee Co-PI:Prof. Hsing-Kuo Kenneth Pao
Overview
M2M networks have been extensively used in many applications. Embedding a good anomaly detection mechanism for the protection of an M2M network application has drawn attentions from researchers in related fields. The various anomalies in M2M applications might be caused by sensor malfunction, sensor fault, compromised sensors, or signs of possible attacks on the sensors or on the communication channels. An anomaly detection mechanism is important to monitor whether or not the sensors are working properly and enhance the streaming data quality that is fed in by sensors. Furthermore, one would expect a useful anomaly detector to identify potential anomalies quickly or in real-time. This is the primary research challenge to enable anomaly detection in M2M networks.
Challenges
- Correctly profile “normal” behaviors, and define and identify “clean” data.
- Distinguish the noise data and anomalies especially when the data are collected in harsh environment.
- Detect contextual anomaly and understand the spatial and temporal dependency of sensor data.
- Build a lightweight anomaly detector under resource constraints, such as CPU, memory storage and battery limitation.
- Collect the data by the front-end detectors in a distributed fashion.
- In-time anomaly detection to enable real-time response, so that service availability is maintained.
- Customization of anomaly detector under different environments.
- Understand evolution of the anomaly/normal behavior over time and detect anomalies in an adversarial environment
Goals
The goal of this project is to build an anomaly detection system for M2M applications. The system consists of a rich set of detection algorithms that are suitable for various M2M applications. The tasks of this project are as follows.
- Build a general framework of anomaly detection system for M2M applications
- Complete an anomaly detection toolbox
- Develop an SOP of anomaly detection mechanism for a new M2M network application
Members
Publications
H. Pao, Y. Lee and C. Huang, "Statistical Learning Methods for Information Security: Fundamentals and Case Studies", Appl. Stoch. Model. Bus. Ind., vol. 31, no. 2, mar 2015, pp. 97–113.
W. Lai et al., "A Distributed Ensemble Scheme for nonlinear Support Vector Machine", in 2015 IEEE Tenth International Conference on Intelligent Sensors, Sensor Networks and Information Processing (ISSNIP), pp. 1-6.
W. Lai et al., "A Distributed Ensemble Scheme for nonlinear Support Vector Machine", in 2015 IEEE Tenth International Conference on Intelligent Sensors, Sensor Networks and Information Processing (ISSNIP), pp. 1-6.
E. N. Jannah, H. Pao, "Sensor Reading Prediction Using Anisotropic Kernel Gaussian Process Regression", in 2014 IEEE International Conference on Internet of Things (iThings), and IEEE Green Computing and Communications (GreenCom) and IEEE Cyber, Physical and Social Computing (CPSCom), pp. 207-214.
Y. Shen et al., "Continuous Monitoring and Distributed Anomaly Detection for Ambient Factors", in 2014 IEEE International Conference on Internet of Things (iThings), and IEEE Green Computing and Communications (GreenCom) and IEEE Cyber, Physical and Social Computing (CPSCom), pp. 31-38.
L. C. Jaw, Y. Lee, "Engine diagnostics in the eyes of machine learning", in Ceramics; Controls, Diagnostics and Instrumentation; Education; Manufacturing Materials and Metallurgy, American Society of Mechanical Engineers (ASME), 2014.
L. Xu et al., "A Hierarchical Framework Using Approximated Local Outlier Factor for Efficient Anomaly Detection", Procedia Computer Science, vol. 19, 2013, pp. 1174-1181.
H. Pao, Y. Chou and Y. Lee, "Malicious URL Detection Based on Kolmogorov Complexity Estimation", in 2012 IEEE/WIC/ACM International Conferences on Web Intelligence and Intelligent Agent Technology, pp. 380-387.
H. Pao, Y. Chou and Y. Lee, "Malicious URL Detection Based on Kolmogorov Complexity Estimation", in 2012 IEEE/WIC/ACM International Conferences on Web Intelligence and Intelligent Agent Technology, pp. 380-387.
Y. Lee, Y. Yeh and Y. F. Wang, "Anomaly Detection via Online Oversampling Principal Component Analysis", IEEE Transactions on Knowledge and Data Engineering, vol. 25, no. 7, 2013, pp. 1460-1470.
National Yang Ming Chiao Tung University
