Anomaly detection in OT: building the baseline and managing false positives

Why OT should be the ideal environment for anomaly detection

Anomaly detection rests on a simple principle: establish what is normal, then detect deviations. The more predictable the environment, the better this technique works: the baseline is stable, the deviations are sharp, and the signal stands out easily from the noise.

From this point of view, OT environments look like an ideal case. A production line running at full regime has extremely repetitive communication patterns: PLC A reads values from the I/O every 100ms, the SCADA server polls the PLCs every 5 seconds, the historian collects data every minute. These cycles repeat with minimal variation for weeks, months, years. The communication matrix (which devices talk to which others, on which ports, at which frequency) is stable and documentable.

This predictability is both the strength of anomaly detection in OT and, as we will see, the main source of its difficulties. Because the environment is predictable in normal operating mode, but it is subject to legitimate exceptional events that on their own are enough to saturate an alerting system that is not correctly calibrated.

The problem of legitimate anomalous behavior

Every production plant has recurring events that break the normal patterns in a perfectly legitimate way. The fact that they are legitimate does not make them less "anomalous" from the point of view of a detection system observing network traffic.

Scheduled maintenance: during a maintenance shutdown, communication patterns change radically. Technicians connect to devices they did not communicate with. New parameters are loaded onto the PLCs. Tests are run that generate unusual traffic. An uncalibrated anomaly detection system will generate dozens or hundreds of alerts during every maintenance window.

Firmware and software updates: updating an HMI's software or a field device's firmware generates traffic that does not exist in normal operating mode. If the update happens outside the usual maintenance windows, for example for an urgent patch, the pattern is doubly anomalous.

Tests and commissioning: when a new machine is added or a line is modified, the commissioning phase generates traffic completely different from the baseline. New devices appear on the network, new communications are established, technicians move across the network in atypical ways.

Seasonal variations: in some manufacturing sectors, production volumes vary significantly over the year. A change in production rhythm translates into changes in communication patterns: different polling cycles, different data volumes, different operation frequencies.

Extraordinary operations: urgent out-of-hours production, double shifts, non-standard process tests. All legitimate events that break the normal patterns.

Each of these events, if the anomaly detection system is not aware of it, generates alerts. A system that generates dozens of alerts for every scheduled maintenance, in a plant that runs weekly maintenance, stops being useful very quickly. Analysts learn to ignore alerts during maintenance shutdowns, which creates exactly the window of opportunity an attacker might want to exploit.

How to build an accurate baseline

Building a reliable baseline for an OT environment requires a methodical approach that goes beyond simply "observe the traffic for two weeks."

The observation period must be long enough. A two-week period captures daily and weekly patterns, but not monthly or seasonal cycles. For a plant with significant monthly cadences (month-end, inventory closing, planned monthly maintenance), at least six to eight weeks of observation are needed before the baseline is representative. For environments with marked seasonal variations, the optimal period is even longer.

The baseline must distinguish temporal profiles. A baseline that treats Monday-morning traffic and Saturday-night traffic the same way will be inaccurate for both. A sophisticated baseline model keeps traffic profiles separate by time band and day of the week, building an image of "normal" that mirrors the plant's real temporal structure.

Documented exceptions must be explicitly integrated. The scheduled-maintenance calendar must be integrated with the detection system: during maintenance windows, some alerts are suppressed or their confidence level is lowered. This does not mean ignoring maintenance: it means not generating hundreds of alerts for foreseen and planned events.

The baseline must be specific per host pair, not generic per network. A baseline that treats the entire OT network as a single statistical blob loses the granularity needed to detect anomalous behavior of individual devices. The meaningful baseline answers questions like: "has the line-3 PLC ever communicated directly with the historian server?" or "is it normal for the HMI panel in the control room to connect to this subnet?".

The false-positive rate as the primary metric

In OT anomaly detection projects, the false-positive rate is the metric that most often determines success or failure. Not the number of true positives, which in the first weeks of deployment is often low simply because no attack occurs, but the number of false positives analysts have to process every day.

A system that generates fifty alerts a day, of which forty-five are false positives, is a dysfunctional system. Analysts quickly learn that most alerts are not worth the time to investigate, develop informal heuristics to decide what to ignore, and in the process may lose the five real alerts in the noise.

The realistic target for a mature OT anomaly detection system varies with context, but in general a system that requires manual investigation of more than ten to fifteen alerts per day per analyst is already near the limit of sustainability. Below that threshold, every alert receives adequate attention. Above it, you enter triage mode where the quality of analysis degrades.

Reaching an acceptable false-positive rate takes time: the first weeks after deploying a new anomaly detection system are always characterized by high volumes. Improvement happens through systematic analysis of the alerts (what is triggering this alert? is it a pattern that should have been included in the baseline?) and the progressive refinement of the rules and the baseline itself.

Integration with the operational calendar

Integration between the monitoring system and the plant's operational calendar is one of the practical measures with the greatest impact on detection quality.

The principle is simple: the detection system must know what is planned. If maintenance is scheduled for Saturday from 6 to 14, the system must apply different rules during that time window: not ignore the alerts, but contextualize them differently, lower the priority of "new communications" class alerts and raise the priority of "communications toward external destinations" class alerts (which during planned maintenance have no justification).

This integration requires a process: someone must enter the maintenance windows into the system before they happen, and someone must verify that the system received the information correctly. It is not full automation, it requires operational discipline, but the benefit in terms of false-positive reduction is significant.

A more advanced level of integration involves correlation with the ERP or production-planning system to automatically detect changes in shifts and production rhythms, and adapt alerting thresholds accordingly. This further reduces noise during operational transitions.

When a system does not converge

Some OT environments present a specific challenge: they are structurally noisy, that is, they have a level of variability in communication patterns that makes it difficult to establish a stable baseline. The most common causes are environments with frequent production changeovers (multi-product with different setups), plants with many legacy devices that behave irregularly by design, or poorly segmented OT networks where traffic of different natures mixes on the same segments.

In these cases, pure anomaly detection, based solely on deviation from the baseline, is not sufficient on its own. It must be integrated with detection rules based on specific signatures and behaviors: not "this traffic is different from yesterday" but "this device is using Modbus write function codes against a register it has never written in six months." Signature-based rules are less dependent on baseline stability and more robust in variable environments.

The combination of anomaly detection and signature-based detection rules is the most robust approach in most real environments: anomaly detection catches the deviations the rules do not foresee, the rules catch the specifically dangerous behaviors regardless of the baseline context.

The MON5 angle

The quality of an anomaly detection system is decided on the baseline, and this is where MON5's PROTECT phase concentrates: the passive out-of-band NDR observes native OT protocols (Modbus, Siemens S7, OPC UA, PROFINET, EtherNet/IP) and machine learning models build communication profiles per host pair, not per whole network.

The prerequisite, however, is knowing the environment: the DISCOVER phase collects traffic and topology without touching the devices, providing the basis on which to calibrate detection. If you want to understand which baseline your plant starts from, the first step is an OT assessment.