Kaspersky Unified Monitoring and Analysis Platform

Collector

A collector is an application component that receives messages from event sources, processes these messages, and sends them to a storage, correlator, and/or third-party services to identify alerts.

For each collector, one connector and one normalizer must be configured. You can also configure any number of additional normalizers, filters, enrichment rules, and aggregation rules. For the collector to send normalized events to other services, you must add destinations. Normally, two destinations are used: a storage and a correlator.

The collector iterates over the following steps:

1. Receive messages from event sources

   To receive messages, you must configure an active or passive connector. A passive connector only listens for messages from an event source, while an active connector actively polls an event source, such as a database management system.

   Connectors can have different types. The choice of connector type depends on the transport protocol used for messaging. For example, if your event source sends messages over TCP, you must install a connector of the TCP type.

   The application has the following connector types available:

   • tcp
   • udp
   • netflow
   • sflow
   • nats-jetstream
   • kafka
   • kata/edr
   • http
   • sql
   • file
   • 1c-xml
   • 1c-log
   • diode
   • ftp
   • nfs
   • vmware
   • wmi
   • wec
   • snmp-trap
   • elastic
   • etw
2. Parse and normalize events

   Events received by the connector are processed using a normalizer and normalization rules set by the user. The choice of normalizer depends on the format of messages coming from the event source. For example, if your event source sends messages in the CEF format, you must select a normalizer of the CEF type.

   The following normalizers are available in the application:

   • JSON
   • CEF
   • Regexp
   • Syslog (as per RFC3164 and RFC5424)
   • CSV
   • Key-value
   • XML
   • NetFlow (the same normalizer for NetFlow v5, NetFlow v9 and IPFIX)
   • NetFlow v5
   • NetFlow v9
   • SQL
   • IPFIX (v10)
3. Filter normalized events

   You can configure filters to identify events that satisfy certain conditions and only send such events for processing.

4. Enrich and convert normalized events

   Enrichment rules let you add supplementary information from internal and external sources to the events. The application can use the following enrichment sources:

   • constants
   • cybertrace
   • dictionaries
   • dns
   • events
   • ldap
   • templates
   • timezone data
   • geographic data

   Conversion rules let you convert the values of event fields in accordance with certain criteria. The application offers the following conversion methods:

   • lower: convert all characters to lower case.
   • upper: convert all characters to upper case.
   • regexp: extract a substring using RE2 regular expressions.
   • substring: extract a substring by giving its first and last characters.
   • replace: replace some text with a string.
   • trim: delete the specified characters.
   • append: add characters to the end of the field value.
   • prepend: adds characters to the beginning of the field value.
5. Aggregate normalized events

   You can configure aggregation rules to avoid sending many events of the same kind to the storage and/or correlator. Aggregation rules let you combine multiple events into one event. This can help reduce the load on the services responsible for further event processing, conserve storage space and the events per second (EPS) allowance of your license. For example, if you have many events for network connections between two IP addresses that use the same transport and application layer protocols, you can roll up such events for a certain period into one big event.

6. Send out normalized events

   Having passed through all processing steps, the event is sent to the configured destinations.