A Mechanism to Assess the Effectiveness Anomaly Detectors in Industrial Control Systems

The rise in attacks on Industrial Control Systems (ICS) makes it imperative for the anomaly detection mechanisms (ADMs) to be complete with respect to a set of attacks. In this work, a method is proposed to create and launch simulated attacks on ICS. In the proposed method, referred to as ICS Resilience (ICSRes), attacks are generated using a tool suite named A6. A6 mutates data exchanged between any two PLCs connected via the communications network as well as between a PLC and the sensors and actuators connected to it via a Remote Input/Output (RIO) unit. It consists of both single-point and multi-point mutations that can be manipulated in static or in dynamic form. A two-part case study was conducted to assess the effectiveness and completeness of ICSRes and A6 when compared with that of launching humanly designed attacks. Effectiveness is defined as the ability to detect complex attacks that causes process anomalies and completeness refers to the ability to detect the type of attack. In Part I of the study, the attacks were automatically generated and launched using A6. In Part II a set of attacks was generated and launched manually while participating in an international cyber-exercise. In both parts of the study three ADMs, installed in an operational water treatment testbed, were used to assess their completeness with respect to the generated attacks. The results demonstrate the effectiveness of ICSRes and the tools in highlighting the strength and weaknesses of the ADMs and the value of using A6.