Kaspersky Unified Monitoring and Analysis Platform
- Kaspersky Unified Monitoring and Analysis Platform Help
- About Kaspersky Unified Monitoring and Analysis Platform
- Program architecture
- Program licensing
- About the End User License Agreement
- About the license
- About the License Certificate
- About the license key
- About the key file
- About the license code
- Data provision in Kaspersky Unified Monitoring and Analysis Platform
- Adding a license key to the program web interface
- Viewing information about an added license key in the program web interface
- Removing a license key in the program web interface
- Administrator's guide
- Installing and removing KUMA
- Program installation requirements
- Ports used by KUMA during installation
- Reissuing internal CA certificates
- Modifying the self-signed web console certificate
- Synchronizing time on servers
- About the inventory file
- Installation on a single server
- Distributed installation
- Distributed installation in a high availability configuration
- KUMA backup
- Modifying the configuration of KUMA
- Updating previous versions of KUMA
- Troubleshooting update errors
- Delete KUMA
- Working with tenants
- Managing users
- KUMA services
- Services tools
- Service resource sets
- Creating a storage
- Creating a correlator
- Creating an event router
- Creating a collector
- Predefined collectors
- Creating an agent
- Configuring event sources
- Configuring receipt of Auditd events
- Configuring receipt of KATA/EDR events
- Configuring the export of Kaspersky Security Center events to the KUMA SIEM system
- Configuring receiving Kaspersky Security Center event from MS SQL
- Creating an account in the MS SQL database
- Configuring the SQL Server Browser service
- Creating a secret in KUMA
- Configuring a connector
- Configuring the KUMA Collector for receiving Kaspersky Security Center events from an MS SQL database
- Installing the KUMA Collector for receiving Kaspersky Security Center events from the MS SQL database
- Configuring receipt of events from Windows devices using KUMA Agent (WEC)
- Configuring audit of events from Windows devices
- Configuring centralized receipt of events from Windows devices using the Windows Event Collector service
- Granting permissions to view Windows events
- Granting permissions to log on as a service
- Configuring the KUMA Collector for receiving events from Windows devices
- Installing the KUMA Collector for receiving events from Windows devices
- Configuring forwarding of events from Windows devices to KUMA using KUMA Agent (WEC)
- Configuring receipt of events from Windows devices using KUMA Agent (WMI)
- Configuring receipt of DNS server events using the ETW agent
- Configuring receipt of PostgreSQL events
- Configuring receipt of IVK Kolchuga-K events
- Configuring receipt of CryptoPro NGate events
- Configuring receipt of Ideco UTM events
- Configuring receipt of KWTS events
- Configuring receipt of KLMS events
- Configuring receipt of KSMG events
- Configuring the receipt of KICS for Networks events
- Configuring receipt of PT NAD events
- Configuring receipt of events using the MariaDB Audit Plugin
- Configuring receipt of Apache Cassandra events
- Configuring receipt of FreeIPA events
- Configuring receipt of VipNet TIAS events
- Configuring receipt of Nextcloud events
- Configuring receipt of Snort events
- Configuring receipt of Suricata events
- Configuring receipt of FreeRADIUS events
- Configuring receipt of VMware vCenter events
- Configuring receipt of zVirt events
- Configuring receipt of Zeek IDS events
- Configuring Windows event reception using Kaspersky Endpoint Security for Windows
- Configuring receipt of Codemaster Mirada events
- Configuring receipt of Postfix events
- Configuring receipt of CommuniGate Pro events
- Configuring receipt of Yandex Cloud events
- Configuring receipt of MongoDB events
- Monitoring event sources
- Managing assets
- Adding an asset category
- Configuring the table of assets
- Searching assets
- Exporting asset data
- Viewing asset details
- Adding assets
- Assigning a category to an asset
- Editing the parameters of assets
- Archiving assets
- Deleting assets
- Updating third-party applications and fixing vulnerabilities on Kaspersky Security Center assets
- Moving assets to a selected administration group
- Asset audit
- Custom asset fields
- Critical information infrastructure assets
- Integration with other solutions
- Integration with Kaspersky Security Center
- Configuring Kaspersky Security Center integration settings
- Adding a tenant to the list for Kaspersky Security Center integration
- Creating Kaspersky Security Center connection
- Editing Kaspersky Security Center connection
- Deleting Kaspersky Security Center connection
- Importing events from the Kaspersky Security Center database
- Kaspersky Endpoint Detection and Response integration
- Integration with Kaspersky CyberTrace
- Integration with Kaspersky Threat Intelligence Portal
- Integration with R-Vision Security Orchestration, Automation and Response
- Integration with Active Directory, Active Directory Federation Services and FreeIPA
- Connecting over LDAP
- Enabling and disabling LDAP integration
- Adding a tenant to the LDAP server integration list
- Creating an LDAP server connection
- Creating a copy of an LDAP server connection
- Changing an LDAP server connection
- Changing the data update frequency
- Changing the data storage period
- Starting account data update tasks
- Deleting an LDAP server connection
- Authentication using domain accounts
- Connecting over LDAP
- NCIRCC integration
- Integration with the Security Orchestration Automation and Response Platform (SOAR)
- Kaspersky Industrial CyberSecurity for Networks integration
- Integration with Neurodat SIEM IM
- Kaspersky Automated Security Awareness Platform
- Sending notifications to Telegram
- UserGate integration
- Integration with Kaspersky Web Traffic Security
- Integration with Kaspersky Secure Mail Gateway
- Importing asset information from RedCheck
- Configuring receipt of Sendmail events
- Integration with Kaspersky Security Center
- Managing KUMA
- Working with geographic data
- Downloading CA certificates
- Installing and removing KUMA
- User guide
- KUMA resources
- Operations with resources
- Destinations
- Normalizers
- Aggregation rules
- Enrichment rules
- Correlation rules
- Filters
- Active lists
- Viewing the table of active lists
- Adding active list
- Viewing the settings of an active list
- Changing the settings of an active list
- Duplicating the settings of an active list
- Deleting an active list
- Viewing records in the active list
- Searching for records in the active list
- Adding a record to an active list
- Duplicating records in the active list
- Changing a record in the active list
- Deleting records from the active list
- Import data to an active list
- Exporting data from the active list
- Predefined active lists
- Proxies
- Dictionaries
- Response rules
- Notification templates
- Connectors
- Viewing connector settings
- Adding a connector
- Connector settings
- Connector, tcp type
- Connector, udp type
- Connector, netflow type
- Connector, sflow type
- Connector, nats-jetstream type
- Connector, kafka type
- Connector, kata/edr type
- Connector, http type
- Connector, sql type
- Connector, file type
- Connector, 1c-xml type
- Connector, 1c-log type
- [2.0] Connector, diode type
- Connector, ftp type
- Connector, nfs type
- Connector, vmware type
- Connector, wmi type
- Connector, wec type
- Connector, snmp type
- [2.0.1] Connector, snmp-trap type
- Connector, elastic type
- Connector, etw type
- Predefined connectors
- Secrets
- Segmentation rules
- Context tables
- Viewing the list of context tables
- Adding a context table
- Viewing context table settings
- Editing context table settings
- Duplicating context table settings
- Deleting a context table
- Viewing context table records
- Searching context table records
- Adding a context table record
- Editing a context table record
- Deleting a context table record
- Importing data into a context table
- Exporting data from a context table
- Example of incident investigation with KUMA
- Incident conditions
- Step 1. Preliminary steps
- Step 2. Assigning an alert to a user
- Step 3. Check if the triggered correlation rule matches the data of the alert events
- Step 4. Analyzing alert information
- Step 5. False positive check
- Step 6. Determining alert severity
- Step 7. Incident creation
- Step 8. Investigation
- Step 9. Searching for related assets
- Step 10. Searching for related events
- Step 11. Recording the causes of the incident
- Step 12. Incident response
- Step 13. Restoring assets operability
- Step 14. Closing the incident
- Analytics
- Working with events
- Filtering and searching events
- Selecting Storage
- Generating an SQL query using a builder
- Manually creating an SQL query
- Filtering events by period
- Grouping events
- Displaying names instead of IDs
- Presets
- Limiting the complexity of queries in alert investigation mode
- Saving and selecting events filter configuration
- Deleting event filter configurations
- Supported ClickHouse functions
- Viewing event detail areas
- Exporting events
- Configuring the table of events
- Refreshing events table
- Getting events table statistics
- Viewing correlation event details
- Filtering and searching events
- Dashboard
- Reports
- Widgets
- Working with alerts
- Working with incidents
- About the incidents table
- Saving and selecting incident filter configuration
- Deleting incident filter configurations
- Viewing information about an incident
- Incident creation
- Incident processing
- Changing incidents
- Automatic linking of alerts to incidents
- Categories and types of incidents
- Interaction with NCIRCC
- Retroscan
- Working with events
- KUMA resources
- Contacting Technical Support
- REST API
- Creating a token
- Configuring permissions to access the API
- Authorizing API requests
- Standard error
- REST API v1 operations
- Viewing a list of active lists on the correlator
- Import entries to an active list
- Searching alerts
- Closing alerts
- Searching assets
- Importing assets
- Deleting assets
- Searching events
- Viewing information about the cluster
- Resource search
- Loading resource file
- Viewing the contents of a resource file
- Importing resources
- Exporting resources
- Downloading the resource file
- Search for services
- Tenant search
- View token bearer information
- Dictionary updating in services
- Dictionary retrieval
- Viewing custom fields of the assets
- Creating a backup of the KUMA Core
- Restoring the KUMA Core from the backup
- Viewing the list of context tables in the correlator
- Importing records into a context table
- Exporting records from a context table
- REST API v2 operations
- Viewing a list of active lists on the correlator
- Import entries to an active list
- Searching alerts
- Closing alerts
- Searching assets
- Importing assets
- Deleting assets
- Searching events
- Viewing information about the cluster
- Resource search
- Loading resource file
- Viewing the contents of a resource file
- Importing resources
- Exporting resources
- Downloading the resource file
- Search for services
- Tenant search
- View token bearer information
- Dictionary updating in services
- Dictionary retrieval
- Viewing custom fields of the assets
- Creating a backup of the KUMA Core
- Restoring the KUMA Core from the backup
- Viewing the list of context tables in the correlator
- Importing records into a context table
- Exporting records from a context table
- REST API v2.1 operations
- Appendices
- Commands for components manual starting and installing
- Integrity check of KUMA files
- Normalized event data model
- Configuring the data model of a normalized event from KATA EDR
- Alert data model
- Asset data model
- User account data model
- KUMA audit events
- Event fields with general information
- User was successfully signed in or failed to sign in
- User login successfully changed
- User role was successfully changed
- Other data of the user was successfully changed
- User successfully logged out
- User password was successfully changed
- User was successfully created
- User role was successfully assigned
- User role was successfully revoked
- The user has successfully edited the set of fields settings to define sources
- User access token was successfully changed
- Service was successfully created
- Service was successfully deleted
- Service was successfully reloaded
- Service was successfully restarted
- Service was successfully started
- Service was successfully paired
- Service status was changed
- Victoria Metrics alert registered for the service
- Monitoring thresholds changed for the service
- Storage partition was deleted by user
- Storage partition was deleted automatically due to expiration
- Active list was successfully cleared or operation failed
- Active list item was successfully changed, or operation was unsuccessful
- Active list item was successfully deleted or operation was unsuccessful
- Active list was successfully imported or operation failed
- Active list was exported successfully
- Resource was successfully added
- Resource was successfully deleted
- Resource was successfully updated
- Asset was successfully created
- Asset was successfully deleted
- Asset category was successfully added
- Asset category was deleted successfully
- Settings were updated successfully
- Tenant was successfully created
- Tenant was successfully enabled
- Tenant was successfully disabled
- Other tenant data was successfully changed
- Updated data retention policy after changing drives
- The dictionary was successfully updated on the service or operation was unsuccessful
- Response in Active Directory
- Response via KICS for Networks
- Kaspersky Automated Security Awareness Platform response
- KEDR response
- Correlation rules
- Sending test events to KUMA
- Time format
- Mapping fields of predefined normalizers
- Deprecated resources
- Generating events for testing a normalizer
- Information about third-party code
- Trademark notices
- Glossary
Modifying the configuration of KUMA
The KUMA configuration can be modified in the following ways.
- Expanding an all-in-one installation to a distributed installation.
To expand an all-in-one installation to a distributed installation:
- Create a backup copy of KUMA.
- Remove the pre-installed correlator, collector, and storage services from the server.
- In the KUMA web interface, under Resources → Active services, select a service and click Copy ID. On the server where the services were installed, run the service removal command:
sudo /opt/kaspersky/kuma/kuma <collector/correlator/storage> --id <service ID copied from the KUMA web interface> --uninstallRepeat the removal command for each service.
- Then remove the services in the KUMA web interface:
As a result, only the KUMA Core remains on the initial installation server.
- In the KUMA web interface, under Resources → Active services, select a service and click Copy ID. On the server where the services were installed, run the service removal command:
- Prepare the distributed.inventory.yml inventory file and in that file, specify the initial all-in-one initial installation server in the
kuma_coregroup.In this way, the KUMA Core remains on the original server, and you can deploy the other components on other servers. In the inventory file, specify the servers on which you want to install the KUMA components.
Example inventory file for expanding an all-in-one installation to a distributed installation
all:vars:deploy_to_k8s: falseneed_transfer: falsegenerate_etc_hosts: falsedeploy_example_services: falseno_firewall_actions: falsekuma:vars:ansible_connection: sshansible_user: rootchildren:kuma_core:hosts:kuma-core-1.example.com:ip: 0.0.0.0mongo_log_archives_number: 14mongo_log_frequency_rotation: dailymongo_log_file_size: 1Gkuma_collector:hosts:kuma-collector-1.example.com:ip: 0.0.0.0kuma_correlator:hosts:kuma-correlator-1.example.com:ip: 0.0.0.0kuma_storage:hosts:kuma-storage-cluster1-server1.example.com:ip: 0.0.0.0shard: 1replica: 1keeper: 0kuma-storage-cluster1-server2.example.com:ip: 0.0.0.0shard: 1replica: 2keeper: 0kuma-storage-cluster1-server3.example.com:ip: 0.0.0.0shard: 2replica: 1keeper: 0kuma-storage-cluster1-server4.example.com:ip: 0.0.0.0shard: 2replica: 2keeper: 0kuma-storage-cluster1-server5.example.com:ip: 0.0.0.0shard: 0replica: 0keeper: 1kuma-storage-cluster1-server6.example.com:ip: 0.0.0.0shard: 0replica: 0keeper: 2kuma-storage-cluster1-server7.example.com:ip: 0.0.0.0shard: 0replica: 0keeper: 3 - Create and install the storage, collector, correlator, and agent services on other machines.
- After you specify the settings in all sections of the distributed.inventory.yml file, run the installer on the control machine.
sudo ./install.sh distributed.inventory.ymlThis command creates files necessary to install the KUMA components (storage, collectors, correlators) on each target machine specified in distributed.inventory.yml.
- Create storage, collector, and correlator services.
- After you specify the settings in all sections of the distributed.inventory.yml file, run the installer on the control machine.
The expansion of the installation is completed.
- Adding servers for collectors to a distributed installation.
The following instructions describe adding one or more servers to an existing infrastructure to then install collectors on these servers to balance the load. You can use these instructions as an example and adapt them according to your needs.
To add servers to a distributed installation:
- Ensure that the target machines meet hardware, software, and installation requirements.
- On the control machine, go to the directory with the extracted KUMA installer by running the following command:
cd kuma-ansible-installer - Create an inventory file named expand.inventory.yml by copying the expand.inventory.yml.template file:
cp expand.inventory.yml.template expand.inventory.yml - Edit the settings in the expand.inventory.yml inventory file and specify the servers that you want to add in the kuma_collector section.
Example expand.inventory.yml inventory file for adding collector servers
kuma:vars:ansible_connection: sshansible_user: rootchildren:kuma_collector:kuma-additional-collector1.example.comkuma-additional-collector2.example.comkuma_correlator:kuma_storage:hosts: - On the control machine, run the following command as root from the directory with the extracted installer:
./expand.sh expand.inventory.yml
This command creates files for creating and installing the collector on each target machine specified in the expand.inventory.yml inventory file.
- Create and install the collectors. A KUMA collector consists of a client part and a server part, therefore creating a collector involves two steps.
- Creating the client part of the collector, which includes a resource set and the collector service.
To create a resource set for a collector, in the KUMA web interface, under Resources → Collectors, click Add collector and edit the settings. For more details, see Creating a collector.
At the last step of the configuration wizard, after you click Create and save, a resource set for the collector is created and the collector service is automatically created. The command for installing the service on the server is also automatically generated and displayed on the screen. Copy the installation command and proceed to the next step.
- Creating the server part of the collector.
- On the target machine, run the command you copied at the previous step. The command looks as follows, but all parameters are filled in automatically.
sudo /opt/kaspersky/kuma/kuma <storage> --core https://<KUMA Core server FQDN>:<port used by KUMA Core for internal communication (port 7210 by default)> --id <service ID copied from the KUMA web interface> --installThe collector service is installed on the target machine. You can check the status of the service in the web interface under Resources → Active services.
- Run the same command on each target machine specified in the expand.inventory.yml inventory file.
- Creating the client part of the collector, which includes a resource set and the collector service.
- Add the new servers to the distributed.inventory.yml inventory file so that it has up-to-date information in case you need to upgrade KUMA.
Servers are successfully added.
- Adding servers for correlators to a distributed installation.
The following instructions describe adding one or more servers to an existing infrastructure to then install correlators on these servers to balance the load. You can use these instructions as an example and adapt them to your requirements.
To add servers to a distributed installation:
- Ensure that the target machines meet hardware, software, and installation requirements.
- On the control machine, go to the directory with the extracted KUMA installer by running the following command:
cd kuma-ansible-installer - Create an inventory file named expand.inventory.yml by copying the expand.inventory.yml.template file:
cp expand.inventory.yml.template expand.inventory.yml - Edit the settings in the expand.inventory.yml inventory file and specify the servers that you want to add in the kuma_correlator section.
Example expand.inventory.yml inventory file for adding correlator servers
kuma:vars:ansible_connection: sshansible_user: rootchildren:kuma_collector:kuma_correlator:kuma-additional-correlator1.example.comkuma-additional-correlator2.example.comkuma_storage:hosts: - On the control machine, run the following command as root from the directory with the extracted installer:
./expand.sh expand.inventory.ymlThis command creates files for creating and installing the correlator on each target machine specified in the expand.inventory.yml inventory file.
- Create and install the correlators. A KUMA correlator consists of a client part and a server part, therefore creating a correlator involves two steps.
- Creating the client part of the correlator, which includes a resource set and the correlator service.
To create a resource set for a correlator, in the KUMA web interface, under Resources → Correlators, click Add correlator and edit the settings. For more details, see Creating a correlator.
At the last step of the configuration wizard, after you click Create and save, a resource set for the correlator is created and the correlator service is automatically created. The command for installing the service on the server is also automatically generated and displayed on the screen. Copy the installation command and proceed to the next step.
- Creating the server part of the correlator.
- On the target machine, run the command you copied at the previous step. The command looks as follows, but all parameter values are assigned automatically.
sudo /opt/kaspersky/kuma/kuma <storage> --core https://<KUMA Core server FQDN>:<port used by KUMA Core for internal communication (port 7210 by default)> --id <service ID copied from the KUMA web interface> --installThe correlator service is installed on the target machine. You can check the status of the service in the web interface under Resources → Active services.
- Run the same command on each target machine specified in the expand.inventory.yml inventory file.
- Creating the client part of the correlator, which includes a resource set and the correlator service.
- Add the new servers to the distributed.inventory.yml inventory file so that it has up-to-date information in case you need to upgrade KUMA.
Servers are successfully added.
- Adding servers to an existing storage cluster.
The following instructions describe adding multiple servers to an existing storage cluster. You can use these instructions as an example and adapt them to your requirements.
To add servers to an existing storage cluster:
- Ensure that the target machines meet hardware, software, and installation requirements.
- On the control machine, go to the directory with the extracted KUMA installer by running the following command:
cd kuma-ansible-installer - Create an inventory file named expand.inventory.yml by copying the expand.inventory.yml.template file:
cp expand.inventory.yml.template expand.inventory.yml - Edit the settings in the expand.inventory.yml inventory file and specify the servers that you want to add in the 'storage' section. In the following example, the 'storage' section specifies servers for installing two shards, each of which contains two replicas. In the expand.inventory.yml inventory file, you must only specify the FQDN; you will assign the roles of shards and replicas later in the KUMA web interface as you follow the steps of these instructions. You can adapt this example according to your needs.
Example expand.inventory.yml inventory file for adding servers to an existing storage cluster
kuma:vars:ansible_connection: sshansible_user: rootchildren:kuma_collector:kuma_correlator:kuma_storage:hosts:kuma-storage-cluster1-server8.example.comkuma-storage-cluster1-server9.example.comkuma-storage-cluster1-server10.example.comkuma-storage-cluster1-server11.example.com - On the test machine, run the following command as root from the directory with the unpacked installer:
./expand.sh expand.inventory.ymlRunning this command on each target machine specified in the expand.inventory.yml inventory file creates files for creating and installing the storage. - You do not need to create a separate storage because you are adding servers to an existing storage cluster. You need to edit the storage settings of the existing cluster:
- In the Resources → Storages section, select an existing storage and open the storage for editing.
- In the ClickHouse cluster nodes section, click Add nodes and specify roles in the fields for the new node. The following example describes how to specify IDs to add two shards, containing two replicas each, to an existing cluster. You can adapt this example according to your needs.
Example:
ClickHouse cluster nodes
<existing nodes>
FQDN: kuma-storage-cluster1server8.example.com
Shard ID: 1
Replica ID: 1
Keeper ID: 0
FQDN: kuma-storage-cluster1server9.example.com
Shard ID: 1
Replica ID: 2
Keeper ID: 0
FQDN: kuma-storage-cluster1server9.example.com
Shard ID: 2
Replica ID: 1
Keeper ID: 0
FQDN: kuma-storage-cluster1server10.example.com
Shard ID: 2
Replica ID: 2
Keeper ID: 0
- Save the storage settings.
Now you can create storage services for each ClickHouse cluster node.
- To create a storage service, in the KUMA web interface, in the Resources → Active services section, click Add service.
This opens the Choose a service window; in that window, select the storage you edited at the previous step and click Create service. Do the same for each ClickHouse storage node you are adding.
As a result, the number of created services must be the same as the number of nodes being added to the ClickHouse cluster, for example, four services for four nodes. The created storage services are displayed in the KUMA web interface in the Resources → Active services section. Now storage services must be installed on each server by using the service ID.
- Now storage services must be installed on each server by using the service ID.
- In the KUMA web interface, in the Resources → Active services section, select the storage service that you need and click Copy ID.
The service ID is copied to the clipboard; you need it for running the service installation command.
- Compose and run the following command on the target machine:
sudo /opt/kaspersky/kuma/kuma <storage> --core https://<KUMA Core server FQDN>:<port used by KUMA Core for internal communication (port 7210 by default)> --id <service ID copied from the KUMA web interface> --installThe storage service is installed on the target machine. You can check the status of the service in the web interface under Resources → Active services.
- Run the storage service installation command on each target machine listed in the 'storage' section of the expand.inventory.yml inventory file, one machine at a time. On each machine, the unique service ID within the cluster must be specified in the installation command.
- In the KUMA web interface, in the Resources → Active services section, select the storage service that you need and click Copy ID.
- To apply changes to a running cluster, in the KUMA web interface, under Resources → Active services, select the check boxes next to all storage services in the cluster that you are expanding and click Update configuration. Changes are applied without stopping services.
- Specify the added servers in the distributed.inventory.yml inventory file so that it has up-to-date information in case of a KUMA update.
Servers are successfully added to a storage cluster.
- Adding another storage cluster.
The following instructions describe adding an extra storage cluster to an existing infrastructure. You can use these instructions as an example and adapt them to suit your needs.
To add a storage cluster:
- Ensure that the target machines meet hardware, software, and installation requirements.
- On the control machine, go to the directory with the extracted KUMA installer by running the following command:
cd kuma-ansible-installer - Create an inventory file named expand.inventory.yml by copying the expand.inventory.yml.template file:
cp expand.inventory.yml.template expand.inventory.yml - Edit the settings in the expand.inventory.yml inventory file and specify the servers that you want to add in the 'storage' section. In the following example, the 'storage' section specifies servers for installing three dedicated keepers and two shards, each of which contains two replicas. In the expand.inventory.yml inventory file, you must only specify the FQDN; you will assign the roles of keepers, shards, and replicas later in the KUMA web interface by following the steps of these instructions. You can adapt this example to suit your needs.
Example expand.inventory.yml inventory file for adding a storage cluster
kuma:vars:ansible_connection: sshansible_user: rootchildren:kuma_collector:kuma_correlator:kuma_storage:hosts:kuma-storage-cluster2-server1.example.comkuma-storage-cluster2-server2.example.comkuma-storage-cluster2-server3.example.comkuma-storage-cluster2-server4.example.comkuma-storage-cluster2-server5.example.comkuma-storage-cluster2-server6.example.comkuma-storage-cluster2-server7.example.com - On the test machine, run the following command as root from the directory with the unpacked installer:
./expand.sh expand.inventory.ymlThis command creates files for creating and installing the storage on each target machine specified in the expand.inventory.yml inventory file. - Create and install the storage. For each storage cluster, you must create a separate storage, for example, three storages for three storage clusters. A storage consists of a client part and a server part, therefore creating a storage involves two steps.
- Creating the client part of the storage, which includes a resource set and the storage service.
- To create a resource set for a storage, in the KUMA web interface, under Resources → Storages, click Add storage and edit the settings. In the ClickHouse cluster nodes section, specify roles for each server that you are adding: keeper, shard, replica. For more details, see Creating a resource set for a storage.
The created resource set for the storage is displayed in the Resources → Storages section. Now you can create storage services for each ClickHouse cluster node.
- To create a storage service, in the KUMA web interface, in the Resources → Active services section, click Add service.
This opens the Choose a service window; in that window, select the resource set that you created for the storage at the previous step and click Create service. Do the same for each ClickHouse storage.
As a result, the number of created services must be the same as the number of nodes in the ClickHouse cluster, for example, fifty services for fifty nodes. The created storage services are displayed in the KUMA web interface in the Resources → Active services section. Now you need to install storage services on each node of the ClickHouse cluster by using the service ID.
- To create a resource set for a storage, in the KUMA web interface, under Resources → Storages, click Add storage and edit the settings. In the ClickHouse cluster nodes section, specify roles for each server that you are adding: keeper, shard, replica. For more details, see Creating a resource set for a storage.
- Creating the server part of the storage.
- On the target machine, create the server part of the storage: in the KUMA web interface, in the Resources → Active services section, select a storage service and click Copy ID.
The service ID is copied to the clipboard; you will need it for the service installation command.
- Compose and run the following command on the target machine:
sudo /opt/kaspersky/kuma/kuma <storage> --core https://<KUMA Core server FQDN>:<port used by KUMA Core for internal communication (port 7210 by default)> --id <service ID copied from the KUMA web interface> --installThe storage service is installed on the target machine. You can check the status of the service in the web interface under Resources → Active services.
- Run the storage service installation command on each target machine listed in the 'storage' section of the expand.inventory.yml inventory file, one machine at a time. On each machine, the unique service ID within the cluster must be specified in the installation command.
- Dedicated keepers are automatically started immediately after installation and are displayed in the Resources → Active services section with the green status. Services on other storage nodes may not start until services are installed for all nodes in that cluster. Up to that point, services can be displayed with the red status. This is normal behavior when creating a new storage cluster or adding nodes to an existing storage cluster. As soon as the service installation command is run on all nodes of the cluster, all services get the green status.
- Creating the client part of the storage, which includes a resource set and the storage service.
- Specify the added servers in the distributed.inventory.yml inventory file so that it has up-to-date information in case of a KUMA update.
The extra storage cluster is successfully added.
- Removing servers from a distributed installation.
To remove a server from a distributed installation:
- Remove all services from the server that you want to remove from the distributed installation.
- Remove the server part of the service. Copy the service ID in the KUMA web interface and run the following command on the target machine:
sudo /opt/kaspersky/kuma/kuma <collector/correlator/storage> --core https://<KUMA Core server FQDN>:<port used by KUMA Core for internal communication (port 7210 by default)> --id <service ID copied from the KUMA web interface> --install - Remove the client part of the service in the KUMA web interface in the Active services → Delete section.
The service is removed.
- Remove the server part of the service. Copy the service ID in the KUMA web interface and run the following command on the target machine:
- Repeat step 1 for each server that you want to remove from the infrastructure.
- Remove the servers from the relevant sections of the distributed.inventory.yml inventory file to make sure the inventory file has up-to-date information in case you need to upgrade KUMA.
Servers are removed from the distributed installation.
- Remove all services from the server that you want to remove from the distributed installation.
- Removing a storage cluster from a distributed installation.
To remove one or more storage clusters from a distributed installation:
- Remove the storage service on each cluster server that you want to remove from the distributed installation.
- Remove the server part of the storage service. Copy the service ID in the KUMA web interface and run the following command on the target machine:
sudo /opt/kaspersky/kuma/kuma <storage> --id <service ID> --uninstallRepeat for each server. - Remove the client part of the service in the KUMA web interface in the Resources → Active services → Delete section.
The service is removed.
- Remove the server part of the storage service. Copy the service ID in the KUMA web interface and run the following command on the target machine:
- Remove servers from the 'storage' section of the distributed.inventory.yml inventory file to make sure the inventory file has up-to-date information in case you need to upgrade KUMA or modify its configuration.
The cluster is removed from the distributed installation.
- Remove the storage service on each cluster server that you want to remove from the distributed installation.
- Migrating the KUMA Core to a new Kubernetes cluster.
To migrate the KUMA Core to a new Kubernetes cluster:
- Prepare the k0s.inventory.yml inventory file.
The kuma_core, kuma_ collector, kuma_correlator, kuma_storage sections of your k0s.inventory.yml inventory file must contain the same hosts that were used when KUMA was upgraded from version 2.1.3 to version 3.0.3 and then to version 3.2, or when a new installation was performed. In the inventory file, set deploy_to_k8s: true, need_transfer: true. Set deploy_example_services: false.
- Follow the steps for distributed installation using your prepared k0s.inventory.yml inventory file.
Migrating the KUMA Core to a new Kubernetes cluster
When the installer is started with an inventory file, the installer looks for an installed KUMA Core on all hosts where you plan to deploy worker nodes of the cluster. If a Core is found, it is moved from its host to the newly created Kubernetes cluster.
Resolving the KUMA Core migration error
Migration of the KUMA Core from a host to a new Kubernetes cluster may be aborted due to a timeout at the
Deploy Core transfer jobstep. In this case, the following error message is recorded in the log of core-transfer migration tasks:cp: can't stat '/mnt/kuma-source/core/.lic': No such file or directoryTo prevent this error, before you start migrating the KUMA Core:
- Go to the directory with the extracted installer and open the roles/k0s_prepare/templates/core-transfer-job.yaml.j2 file for editing.
- In the core-transfer-job.yaml.j2 file, find the following lines:
cp /mnt/kuma-source/core/.lic {{ core_k0s_home }}/ &&cp /mnt/kuma-source/core/.tenantsEPS {{ core_k0s_home }}/ && - Edit these lines as follows, making sure you keep the indentation (number of space characters):
cp /mnt/kuma-source/core/{{ core_uid }}/.lic {{ core_k0s_home }}/ &&cp /mnt/kuma-source/core/{{ core_uid }}/.tenantsEPS {{ core_k0s_home }}/ && - Save the changes to the file.
You can then restart the distributed installation using the prepared k0s.inventory.yml inventory file. Migrating the KUMA Core from a host to a new Kubernetes cluster will succeed.
If you started migrating the KUMA Core from a host to a new Kubernetes cluster and the migration failed with an error, follow the steps below to fix the error.
To fix the error after attempting to migrate the KUMA Core from a host to a new Kubernetes cluster:
- On any controller of the cluster, delete the Ingress object by running the following command:
sudo k0s kubectl delete daemonset/ingress -n ingress - Check if a migration job exists in the cluster:
sudo k0s kubectl get jobs -n kuma - If a migration job exists, delete it:
sudo k0s kubectl delete job core-transfer -n kuma - Go to the console of a host from the kuma_core group.
- Start the KUMA Core services by running the following commands:
sudo systemctl start kuma-mongodbsudo systemctl start kuma-core-00000000-0000-0000-0000-000000000000 - Make sure that the kuma-core-00000000-0000-0000-0000-000000000000 service has been successfully started:
sudo systemctl status kuma-core-00000000-0000-0000-0000-000000000000 - Make sure that the kuma_core group has access to the KUMA interface by host FQDN.
Other hosts do not need to be running.
- Go to the directory with the extracted installer and open the roles/k0s_prepare/templates/core-transfer-job.yaml.j2 file for editing.
- In the core-transfer-job.yaml.j2 file, find the following lines:
cp /mnt/kuma-source/core/.lic {{ core_k0s_home }}/ &&cp /mnt/kuma-source/core/.tenantsEPS {{ core_k0s_home }}/ && - Edit these lines as follows, making sure you keep the indentation (number of space characters):
cp /mnt/kuma-source/core/{{ core_uid }}/.lic {{ core_k0s_home }}/ &&cp /mnt/kuma-source/core/{{ core_uid }}/.tenantsEPS {{ core_k0s_home }}/ && - Save the changes to the file.
You can then restart the distributed installation using the prepared k0s.inventory.yaml inventory file. The migration of the KUMA Core from a host to a new Kubernetes cluster will succeed.
If the component is not detected on the worker nodes, a clean installation of the KUMA Core is performed in the cluster without migrating resources to it. Existing components must be manually recreated with the new Core in the KUMA web interface.
For collectors, correlators and storages from the inventory file, certificates for communication with the Core inside the cluster will be reissued. This does not change the URL of the Core for components.
On the Core host, the installer does the following:
- Removes the following systemd services from the host: kuma-core, kuma-mongodb, kuma-victoria-metrics, kuma-vmalert, and kuma-grafana.
- Deletes the internal certificate of the Core.
- Deletes the certificate files of all other components and deletes their records from MongoDB.
- Deletes the following directories:
- /opt/kaspersky/kuma/core/bin
- /opt/kaspersky/kuma/core/certificates
- /opt/kaspersky/kuma/core/log
- /opt/kaspersky/kuma/core/logs
- /opt/kaspersky/kuma/grafana/bin
- /opt/kaspersky/kuma/mongodb/bin
- /opt/kaspersky/kuma/mongodb/log
- /opt/kaspersky/kuma/victoria-metrics/bin
- Migrates data from the Core and its dependencies to a network drive within the Kubernetes cluster.
- On the Core host, it moves the following directories:
- /opt/kaspersky/kuma/core → /opt/kaspersky/kuma/core.moved
- /opt/kaspersky/kuma/grafana → /opt/kaspersky/kuma/grafana.moved
- /opt/kaspersky/kuma/mongodb → /opt/kaspersky/kuma/mongodb.moved
- /opt/kaspersky/kuma/victoria-metrics → /opt/kaspersky/kuma/victoria-metrics.moved
After you have verified that the Core was correctly migrated to the cluster, you can delete these directories.
If you encounter problems with the migration, check the logs for records of the 'core-transfer' migration task in the 'kuma' namespace in the cluster (this task is available for 1 hour after the migration).
If you need to perform migration again, you must restore the original names of the /opt/kaspersky/kuma/*.moved directories.
If the /etc/hosts file on the Core host contained lines that were not related to addresses in the 127.X.X.X range, the contents of the /etc/hosts file from the Core host is entered into the coredns ConfigMap when the Core is migrated to the Kubernetes cluster. If the Core is not migrated, the contents of the /etc/hosts file from the host where the primary controller is deployed is entered into the ConfigMap.
- Prepare the k0s.inventory.yml inventory file.