TRACER

The problem​

An intrusion detection system (IDS) watches a network's traffic and raises an alert when something looks like an attack. Modern IDS rely heavily on Machine and Deep Learning algorithms: they are shown examples of normal traffic and attacks, and learn to tell them apart.

This approach, called supervised, simply applies techniques that perform very well in other fields. For anomaly detection, however, it has two limitations:

• Unknown attacks. A model is good at recognising what it saw during training, much less so a new type of attack (a new class). This is called a zero-day attack.
• Traffic changes. New devices, new applications, new habits: even normal traffic evolves over time. The model's performance degrades, and it must be retrained regularly with data freshly labelled by experts. That is slow and expensive.

The idea: an agent that keeps learning​

TRACER is built on reinforcement learning. Instead of learning once during training, an agent makes a decision for each network flow, receives a reward if it is right or a penalty if it is wrong, and adjusts its behaviour as it goes: it trains continuously.

Everything hinges on the reward, which guides the agent's learning. Yet in most existing work, it is computed from the ground truth (the label). In a real network, that label is unknown for live traffic, all the more so for a zero-day.

A two-sided reward​

TRACER picks how to reward the agent depending on what it knows about the flow.

The second path is the heart of our contribution. TRACER keeps in memory a sample of traffic it is sure is benign. To judge a new flow, it clusters a window of the current flow together with that sample of benign flows, in three different ways (Fuzzy C-Means, Fuzzy DBSCAN and Fuzzy HDBSCAN). Each algorithm votes:

• “normal” if the flow falls into a group made mostly of known normal traffic, and clearly belongs to that group (we will see what that means just below).
• “alert” in every other case.

These algorithms are fuzzy: they do not only say which group a flow belongs to, but how strongly it belongs (a membership degree between 0 and 1). If a point sits at the centre of a cluster, its membership degree is close to 1; if it lies on the border between two clusters, its membership degree is low. We use that degree as a confidence measure: a hesitant vote yields a small reward, a clear-cut vote a large one. The agent is therefore never heavily punished or rewarded on the basis of an uncertain opinion.

Using three algorithms rather than one is not a luxury: each sees the traffic from a different angle, and none is reliable alone on every network.

Try it yourself​

Set each algorithm's opinion and the agent's decision: the reward is computed as in the paper.

How we evaluated it​

TRACER is evaluated on four reference datasets (NSL-KDD, CICIDS17, UNSW-NB15 and ToN-IoT), with exactly the same settings. Clustering does not work well in high dimensions, which is why we reduced the dimensionality with PCA, SCARF and an autoencoder, so as to work in lower-dimensional spaces. The evaluation covers two realistic scenarios:

1. The zero-day attack. An entire attack family is removed from training, then made to appear later on. This is repeated for each of the 23 families. It is more demanding than many studies, where the “zero-day” is only a variant of an already known attack. TRACER learns from the labels on known traffic, and switches to the vote-based reward for unknown traffic — including the zero-day attack.
2. Missing labels. Labels are progressively hidden, from 0% to 100%, to see how long the agent holds up.

Performance is measured with a detection score: it is high only if the agent recognises both the attack and normal traffic. A system that raises alerts all the time gets a poor score.

Results: facing never-seen attacks​

The detection score is often around 80% and exceeds 90% for several attack families, while staying strong on already known attacks: learning something new does not erase the old, so catastrophic forgetting is avoided.

Two results stand out:

• Rare attacks are detected. The Man-in-the-Middle attack in ToN-IoT (78.9%), Shellcode in UNSW-NB15 (81.5%) and U2R in NSL-KDD (78.5%) each account for less than 2% of the data.
• The vote-based reward sometimes beats the supervised one. On some attacks (Exploits and Backdoor in UNSW-NB15, U2R in NSL-KDD), TRACER outperforms an agent rewarded with the true labels.

Explore the results attack by attack​

It learns fast​

As soon as the first flows of a new attack show up, the score climbs almost vertically: for most attacks, it reaches 90% of its final value within a few dozen steps. This is essential for a zero-day, where only a handful of examples are available before one has to react.

Results: when labels are missing​

Performance stays stable until about 70% of labels are hidden, then drops. In other words, a small fraction of labelled data is enough to “prime” the agent, and the vote then takes over. With only 30% of labels, the overall score stays around 80% on most datasets.

Simpler than expected​

Reinforcement learning is designed for situations where each decision influences the states that follow, like a game of chess. That is not the case here: raising an alert and classifying a flow changes nothing about the next flow. The problem then boils down to a much simpler form, the contextual bandit, where only the immediate reward matters: complexity is lower and convergence faster.

The paper shows this and then verifies it: a lightweight “bandit” agent reaches performance comparable to the deep reinforcement learning agent on most attacks, with training time almost cut in half.

In the field​

TRACER is designed to run close to the network, on commodity hardware. It has been deployed on an industrial partner's infrastructure, where it was measured with traffic replayed from the four datasets.

Limits and next steps​

TRACER is not infallible. When an attack looks too much like normal traffic, as with Web Attack, Bot and Infiltration in CICIDS17, the clustering algorithms cannot separate it, and the agent raises too many false alerts. This is a limit for a system that only observes flow statistics, not their content.

Next steps:

• combine several data representations, since each one reveals different attacks: we are studying new latent spaces built with other dimensionality reduction techniques.
• weight the three algorithms according to the monitored network and the targeted attacks.
• study gradual traffic drift: the appearance of a zero-day is seen as a sudden concept drift, but traffic can also evolve more gradually.
• connect TRACER to real production traffic.