Source
Contains parameters for a specific event source.
The regular expressions and event normalizing rules specified in the configuration file are grouped by event sources that are represented by Source elements. Usually these event sources are devices that issue events, which afterward are checked by Feed Service. Every Source element contains a set of rules. There can be one or more Source elements in the InputSettings > RegExps element.
Rules workflow
The way Feed Service chooses rules from different Source elements is described in the following flow chart.
Choosing a rule
Note that event normalizing rules are applied first and regular expressions are applied afterwards.
The regular expressions of the default event source for finding URLs, IP addresses, and hashes are universal; that is, they can be used for parsing events issued by most devices. They can be used for parsing events that contain multiple URLs, but cannot be used, for example, for parsing events that contain URLs with no protocol specified. The use of universal regular expressions lowers the performance of Feed Service, compared to using device-specific regular expressions. Also, the universal regular expressions do not handle the dispersal, in an event, of different parts of a URL (for example, the host and the path). The universal regular expressions for finding hashes can extract symbol sequences that actually are not hashes.
Special event sources
There must be an event source with the default identifier (<Source id="default">). Rules of the default event source have lower priority than rules specific to the event source. Rules specific to the event source are applied first. Rules of the default event source are applied next. If a rule specific to the event source and a rule of the default event source have the same name, the rule of the default event source is applied only if the rule specific to the event source had no matches.
There are two special event sources that you can use: http_single_lookup (<Source id="http_single_lookup">) and http_file_lookup (<Source id="http_file_lookup">).
The rules of the http_single_lookup event source are used when single values are searched for by using CyberTrace Web.
The rules of the http_file_lookup event source are used when hashes of specified files or indicators in log files are searched for by using CyberTrace Web. Therefore, if you have to search for values contained in a log file that has a special format, we recommend specifying rules for the http_file_lookup event source.
If the configuration file contains the http_single_lookup event source or the http_file_lookup event source, we strongly recommend that you do not remove the regular expressions specified in these special event sources by default and, instead, edit them as needed.
Path
InputSettings > RegExps > Source
Attributes
This element has the following attributes.
Source element attributes
Attribute |
Description |
|---|---|
|
A unique identifier of the event source. In detection events, the identifier of the event source can be referred to by the %SourceId% pattern. |
|
The IP address of the event source. If an event has arrived from an event source that has the specified IP address, the event is processed by using the rules contained in this The |
|
The host name of the event source. The value of the host name is extracted from the event. In syslog events, the host name follows the timestamp (https://tools.ietf.org/html/rfc5424). For example, in the event If an event has arrived from an event source that has the specified host name and the IP address of the event source is not among those specified in the The hostname attribute cannot be set for the default, |
|
The regular expression that is used to determine if an event comes from the source. The specified regular expression is applied to an event. If the regular expression matches the event one or more times, the event is considered to be from the source. In this case, the event is processed by using the rules contained in this The hostname attribute cannot be set for the |
Nested elements
This element is a container for the following nested elements:
- Regular expressions
Regular expressions that are used to parse incoming events originating from this source.
Each regular expression is a separate element with the name of the regular expression.
- NormalizingRules
Rules for modifying incoming events.
Example
The following is an example of this element.
<Source id="CustomSource" ip="192.0.2.15"> <RE_MD5 type="MD5" extract="all">([\da-fA-F]{32})</RE_MD5> <RE_SHA1 type="SHA1" extract="all">([\da-fA-F]{40})</RE_SHA1> <RE_SHA256 type="SHA256" extract="all">([\da-fA-F]{64})</RE_SHA256> <NormalizingRules> ... </NormalizingRules> </Source> |