Building anomaly detection systems

We build anomaly detection over logs, transactions, and telemetry: we find deviations without pre-written rules and explain what exactly changed.

The system catches atypical behavior in a stream, including failures outside the known list.

Pricing

from ₽1,800,000

per project

This is the lower bound of a project. We scope the exact quote after a brief: it depends on volume and the number of integrations.

What's included

Capabilities

No hand-written rules

The system learns normal from history and spots a deviation no rule describes yet.

Explained finding

Each anomaly comes with the feature that went out of range and by how much.

Fewer false alarms

Threshold and seasonality are tuned so a nightly peak does not wake the on-call engineer.

Alert to a channel

The alert goes to the on-call channel with context and a link to the raw data.

How we work

From request to result

  1. 1

    Stream audit

    We review which events exist and at what rate: normal cannot be built from a sparse stream.

  2. 2

    Feature and normal schema

    We fix features, window, and seasonality, and agree on the allowable false-alarm rate.

  3. 3

    Training and thresholds

    We train the normal model and set the threshold from triage of pilot alerts.

  4. 4

    On-call and triage

    We connect alerts to the on-call channel and start triage: false alarms refine the threshold.

Who it fits

Check if it fits you

Good fit

  • A steady event stream

    Logs, transactions, or telemetry flow continuously: normal is built from enough volume.

  • History exists for normal

    At least one full period is recorded, seasonal peaks included.

Not a fit

  • You need a deterministic rule

    If the decision must be explainable as a rule, a statistical model will not fit.

  • No on-call owner

    Without an on-call owner the system becomes noise that is soon ignored.

Track record

What you get

3×

fewer false alarms after tuning

The system marked atypical operations by feature deviation, and false alarms dropped as it was tuned.

Scope of work

What the delivery includes

A data and baseline audit

A data and baseline audit shows which baseline to treat as normal and which features to combine into a score.

A model and threshold

We train the model on history and tune the threshold in the pilot, so false alarms stay rare and keep attention on real events.

Alert integration

Alert integration delivers events into your bus or messenger with the deviating feature, so the engineer triages them in place.

A pilot with triage

For the first weeks we triage alerts with your on-call engineer and refine the boundary before the system enters the process.

How it works

How the system is built

  1. Normal is built from history

    Normal is built from history: seasonality, working hours, and holidays are folded into the model. A nightly peak lands inside the usual range, so the on-call engineer reacts only to a real deviation.

  2. Features are combined

    Features are combined into one score: the operation amount, a new device, and customer activity combine. A plain operation passes quietly, so an alert comes from a combination that steps outside the learned normal.

  3. Each finding is explained

    Each finding is explained: you see which feature crossed the boundary and by how much. Context travels with the anomaly, so the on-call engineer grasps the cause fast and decides what to do next.

  4. The alert carries the raw data

    The alert carries the raw data, so the on-call engineer triages the event in place. The deviating feature arrives with the raw event, and the response takes less time.

Limitations and cost

Where the solution pushes back

  • 01

    Threshold for new behavior

    New legitimate behavior patterns look anomalous at first, so the threshold is softened. The threshold returns to its working level once normal updates on new data.

  • 02

    A cold start

    A cold start needs an observation period or manual labeling, so the system accumulates history over the first weeks. A cold start is the price of an accurate normal.

  • 03

    Disguise as normal

    Detection catches deviations, so deliberate disguise as normal slips past. Such attacks are hunted with other methods, separately from statistics.

  • 04

    The model needs upkeep

    A change in business or scale shifts normal, and it has to be recomputed. The model needs upkeep: normal is revisited on a schedule and after major changes.

Stack

Technologies

The technologies we use in our work and their role in a project.

Stream and models

The event stream runs through Kafka, aggregates are computed in ClickHouse, models train on PyTorch, alerts are sent by Grafana, and rules are stored in PostgreSQL.

The event bus: the log and transaction stream reaches the detector without losing order.Computes feature aggregates per window: normal and deviation are derived from them.Trains normal models: autoencoders and sequence models catch complex deviations.Dashboard and alerts: the alert arrives in the on-call channel with a chart and context.Stores thresholds, alert history, and triage notes: false alarms are visible from them.
Budget estimate

Cost calculator

This is an indicative range, not a quote. We scope the exact estimate after a brief: it depends on volume and the number of integrations.

Streams and metrics

10

Complexity

Streaming detection

Add-ons

Support

Year one: updates and monitoring, a reply within 2 business days.

Service

Building anomaly detection systems

The estimate covers our team's work. Hardware, third-party service licences and travel are added on request.

Estimated cost

₽2.2M

Timeline

5 wk

Questions

Frequently asked questions

How is this better than threshold rules?

Rules describe known failures. The model catches a deviation from normal for which no rule is written yet.

How explainable are the findings?

Each anomaly carries the feature and the magnitude of deviation. The model does not give the full cause but points where to look.

What about false alarms?

The threshold is tuned in the pilot: for the first weeks we triage alerts with the on-call engineer and refine the boundary.

Can it catch a targeted attack?

Detection catches deviations, not disguise. An attack that looks like normal needs other methods.

How much history is needed to start?

Normal needs an observation period, usually several weeks, to account for seasonality and working hours. With no history we start from manual labeling and move the boundary to the model gradually.

How does it fit the current monitoring?

We deliver alerts into your bus or messenger over a webhook, so the on-call engineer works in a familiar tool. Each event arrives with the deviating feature and raw data for triage.

What to do with a found anomaly?

The on-call engineer triages the event from the attached data: it shows which feature crossed the boundary and by how much. Feedback from triage goes back into the model to refine the threshold.

What if there are too many alerts?

We raise the threshold through triage: if the alert stream outpaces the team, the boundary gets stricter. Some findings go to a digest instead of an urgent channel.

Ready to discuss your task?

Leave a request – we'll get back with a proposal and a preliminary estimate.

Contact us