Connector, file type
Connectors of the file type are used for getting data from text files when working with Windows and Linux agents. One line of a text file is considered to be one event. \n is used as the newline character.
If while creating the collector at the Transport step of the installation wizard, you specified a connector of the file type, at the Event parsing in the Mapping table, you can pass the name of the file being processed by the collector or the path to the file in the KUMA event field. To do this, in the Source column, specify one of the following values:
$kuma_fileSourceNameto pass the name of the file being processed by the collector in the KUMA event field.$kuma_fileSourcePathto pass the path to the file being processed by the collector in the KUMA event field.
When you use a file connector, the new variables in the normalizer will only work with destinations of the internal type.
To read Windows files, you need to create a connector of the file type and manually install the agent on Windows. In one Windows Agent, you can configure multiple connections of different types, but there must be only one file type. The Windows agent must not read its files in the folder where the agent is installed.
We do not recommend running the agent under an administrator account; read permissions for folders/files must be configured for the user account of the agent. We do not recommend installing the agent on important systems; it is preferable to send the logs and read them on dedicated hosts with the agent.
For each file that the connector of the file type interacts with, a state file (states.ini) is created with the offset, dev, inode, and filename parameters. The state file allows the connector, to resume reading from the position where the connector last stopped instead of starting over when rereading the file. Some special considerations are involved in rereading files:
- If the
inodeparameter in the state file changes, the connector rereads the corresponding file from the beginning. When the file is deleting and recreated, theinodesetting in the associated state file may remain unchanged. In this case, when rereading the file, the connector resumes reading in accordance with theoffsetparameter. - If the file has been truncated or its size has become smaller, the connector start reading from the beginning.
- If the file has been renamed, when rereading the file, the connector resumes reading from the position where the connector last stopped.
- If the directory with the file has been remounted, when rereading the file, the connector resumes reading from the position where the connector last stopped. You can specify the path to the files with which the connector interacts when configuring the connector in the File path field.
Settings for a connector of the file type are described in the following tables.
Basic settings tab
Setting |
Description |
|---|---|
Name |
Unique name of the resource. The maximum length of the name is 128 Unicode characters. Required setting. |
Tenant |
The name of the tenant that owns the resource. Required setting. |
Type |
Connector type: file. Required setting. |
Path to the file. |
The full path to the file that the connector interacts with. For example,
Required setting. |
Update timeout, sec |
The time in seconds for which the file must not be updated for KUMA to apply the action specified in the Timeout action drop-down list to the file. Default value: The entered value must not be less than the value that you entered on the Advanced settings in the Poll interval, sec field. |
Timeout action |
The action that KUMA applies to the file after the time specified in the Update timeout, sec:
|
Auditd |
This toggle switch enables the auditd mechanism to group auditd event lines received from the connector into an auditd event. If you enable this toggle switch, you cannot select a value in the Delimiter drop-down list because \n is automatically selected for the auditd mechanism. If you enable this toggle switch in the connector settings of the agent, you need to select \n in the Delimiter drop-down list in the connector settings of the collector to which the agent sends events. The maximum size of a grouped auditd event is approximately 4,174,304 characters. KUMA classifies Auditd events in accordance with the algorithm. For example, suppose the following records were received for processing:
The algorithm gives one single-line event of the LOGIN type (because the LOGIN type has code 1006 and it is less than 1300, which is the code of AUDIT_FIRST_EVENT), and one multi-line event with SYSCALL and PROCTITLE. |
Description |
Description of the resource. The maximum length of the description is 4000 Unicode characters. |
Advanced settings tab
Setting |
Description |
|---|---|
Debug |
The switch enables resource logging. The toggle switch is turned off by default. |
Buffer size |
Buffer size in bytes for accumulating events in the RAM of the server before sending them for further processing or storage. The value must be a positive integer. Default buffer size: 1,048,576 bytes (1 MB). Maximum buffer size: 67,108,864 bytes (64 MB). |
Number of handlers |
Number of handlers that the service can run simultaneously to process response rules in parallel. To determine the number of handlers, you can use the following formula: (<number of CPUs> / 2) + 2. The value must be a positive integer up to 999. |
Poll interval, ms |
The interval in milliseconds at which the connector rereads files in the directory. Default value: The entered value must not be less than the value that you entered on the Basic settings in the Update timeout, sec field. We recommend entering a value less than the value that you entered in the Event buffer TTL field because this may adversely affect the performance of Auditd. |
Character encoding |
Character encoding. The default is UTF-8. |
Event buffer TTL |
Buffer lifetime for auditd event lines, in milliseconds. Auditd event lines enter the KUMA collector and accumulate in the buffer. This allows multiple auditd event lines to be grouped into a single auditd event. The buffer lifetime countdown begins when the first auditd event line is received or when the previous buffer lifetime expires. Possible values: 700 to 30,000. The default value is This field is available if you have enabled the Auditd toggle switch on the Basic settings tab. The auditd event lines accumulated in the buffer are kept in the RAM of the server. We recommend caution when increasing the buffer size because memory usage by the KUMA collector may become excessive. You can verify how much server RAM the KUMA collector is using in KUMA metrics. If you want a buffer lifetime to exceed 30,000 milliseconds, we recommend using a different auditd event transport. For example, you can use an agent or pre-accumulate auditd events in a file, and then process this file with the KUMA collector. |
Transport header |
Regular expression for auditd events, which is used to identify auditd event lines. You can use the default value or edit it. The regular expression must contain the You can revert to the default regular expression for auditd events by clicking Reset to default value. |