Configuring Elasticsearch destinations

Before your connections can successfully send data to a destination, your Tanium Cloud instance, CMP network egress allow list, and network allow list must be configured. Note the following:

  • Sign in to the CMP and configure a network egress allow list rule for each destination fully qualified domain name (FQDN) and associated port. For more information on configuring the network egress allow list, see Tanium Cloud Deployment Guide: Configuring network egress allow list rules in the CMP.
  • Tanium Cloud does not support non-TLS plaintext HTTP URLs.
  • Tanium does not support sending data over TCP port 25 outbound. If you create a rule with external access for an SMTP email server destination (default TCP port 465 or TCP port 587), you can only associate the port with 1 FQDN.

  • For other destinations, you can reuse a port for multiple destination FQDNs.

  • Your Tanium Cloud instance has a proxy cluster with 2 public IP addresses. If a destination is in your network, add inbound traffic from these IP addresses to your network allow list.

For more information, see Tanium Cloud Deployment Guide: Network egress. For assistance, contact Tanium Support.

With Elasticsearch, you can search, analyze, and get actionable insights in real time from almost any type of structured and unstructured data source. With Connect, Tanium can write data directly to Elasticsearch.

Specify general connection information

  1. On the Connect Overview page, scroll to the Connections section and click Create Connection.
  2. Enter a name and description for the connection.
  3. (Optional) In the General Information section, expand Advanced to configure the following settings:

    Log Level

    By default, Log Level is set to Information. To reduce the amount of logging, you can set Log Level to Warning, Error, or Fatal.


    Override Log Level

    If you are debugging the connection, select Override Log Level to set a Temporary Log Level (such as Trace or Debug) on this connection for a selected Number of Runs (up to 24). A scheduled or manual connection run, once started, counts towards the number of runs, regardless of the connection status. After the number of runs elapse, the logging for this connection returns to the Log Level you selected to prevent finer-grained logging from consuming additional resources for an indefinite number of runs.

    Minimum Pass Percentage

    Minimum percentage of the expected rows that must be processed for the connection to succeed.

    Memory Ceiling (GB)

    Maximum memory for the node process to run the connection. This defaults to 1 GB per connection, and cannot exceed the global maximum sum of memory for all running connections (by default, 8 GB). Increase this setting if a connection frequently exhibits out of memory errors while running.

    If the sum of simultaneously scheduled connection Memory Ceiling values exceed the global Memory Ceiling, connections run until the global Memory Ceiling is reached, then any remaining connections enter a waiting queue if you select the Queue Connections configuration setting, or fail if you clear the Queue Connections configuration setting.

Configure the connection source

The connection source determines what data you are sending to the destination. This data is usually information from Tanium, such as a saved question, question log, client status, or event. The settings vary depending on which source you choose.




If you want to use nested JSON for your Elasticsearch data, deselect the Flatten Results setting in the Advanced section for the source. If you leave the data flattened, a single JSON item with all of the rows is returned.

Configure Elasticsearch destination

  1. In the Configuration section, provide a name for the destination.
    • Specify a unique name to save the configuration information as a new destination. Select New, and then enter a Destination Name.

    • Select an existing destination. Select Existing, and then select a destination from the Destination Name drop-down list. If you edit the settings for an existing destination, all connections that use that destination are affected.

    • Copy an existing destination. Select New, and then click Copy Settings. Select a destination from the drop-down list, click Apply, and update the Destination Name.

  2. Define a URL to access the Elasticsearch API.
    You can use a variable for the URL, as listed in Reference: Variables. The URL is in the following format:
    <protocol>://<elasticSearchHost>:<elasticSearchPort>/<index>/_bulk
    where:

    protocol

    Specifies the protocol for accessing the Elasticsearch server (http or https). If you specify https for the URL, tThe connection validates the Transport Layer Security (TLS) certificate. You can also select Trust on First Use to accept the certificate presented from the server and trust only that certificate for future connection runs.

    elasticSearchHost

    Specifies the fully qualified domain name (FQDN) or IP of the Elasticsearch server.

    elasticSearchPort

    Specifies the port on which the Elasticsearch server is listening. The default is: 9200.

    index

    Specifies the unique identifier for this API request. The common format for this identifier is: {name}-{0:yyyy.MM.dd}. This date format creates a unique index each day.

    If you submit an external access request for traffic, your configured destination FQDN or IP address, port, and protocol must match the FQDN or IP address, port, and protocol submitted in the external access request.

  3. In the Advanced section for the destination, select Use Authentication if you want to use authentication with Elasticsearch, then enter a User Name and Password.

    If you specify http for the URL, authentication information is encrypted at rest, but passed unencrypted to the destination.

  4. In the Advanced section, define the Batch Size, which determines the number of messages to batch before sending data to Elasticsearch, and the connection Timeout in seconds.
  5. If you have a proxy configured for your Tanium Module Server, select Use Tanium Module Server Proxy Setting. When selected, the proxy makes the outbound connection to the specified Elasticsearch instance provided in the URL.

Configure filters

(Optional) In the Configure Output > Filters section, you can specify filters to modify the data that you are getting from your connection source before it is sent to the destination.

For more information about the types of filters you can configure, see Reference: Filtering options.

Format data for Elasticsearch

In the Configure Output > Format > Advanced section, choose the columns that you want to use as an _id metadata value in Elasticsearch. The _id value is a hash value generated from the combination of the selected column values, before applying any configured Value Escape Regex regular expressions and Value Escape Replace value substitutions. If you select at least one column, the connection data includes an additional column (id) which also contains the _id hash value. If you do not select any columns, Elasticsearch generates a unique _id value.



In the Configure Output > Format > Advanced section, choose the Column names for hashed _id metadata value used in Elasticsearch. The _id value is a hash value generated from the combination of the selected column values, before applying any configured Value Escape Regex regular expressions and Value Escape Replace value substitutions. If you select at least one column, the connection data includes an additional column (id) which also contains the _id hash value. If you do not select any columns, Elasticsearch generates a unique _id value.



Select columns whose values generally remain static, such as ComputerId, ComputerName, or ComputerSerialNumber.

If a saved question is selected as a source, select Group columns by sensor to return the results as an array of objects instead of an array of simple value types. You cannot customize the columns with this option.

In the Configure Output > Columns section, you can change the Destination Label of each column and Value Type to force the column to be a String, Numeric, or Date/Time value.

If you choose Numeric for the value, you can specify a default value that is used if the data cannot be coerced into a numeric value. You can specify any negative or positive number.

If you choose Date/Time for the value, specify the format that you want to use for the column. For more information about using a variable, see Time stamp variables.

For more information about column customizations, see Reference: Column customizations.

Schedule the connection

Connections can run at a highly configurable time interval, such as multiple times per hour, day, week, or month.

If you do not enable the schedule, the connection only runs when you manually run it, unless you configure an Event source. Connections with Event sources only run when a configured event is detected, and cannot be scheduled or manually run.

Use the Schedule section to update the schedule:

  • Select Enable schedule.
  • In the Schedule Type, select Basic to build a schedule with the provided controls.
  • To view or edit the Cron expression directly, select Advanced - Define as a Cron Expression, and use the Advanced field to edit the Cron expression.




If a user that owns a scheduled connection is deleted, future scheduled instances of that connection do not run. For more information, see Problem: Scheduled connection owned by a deleted user no longer runs.

For more information about Cron syntax, see Reference: Cron syntax.

Save and verify connection

  1. After you enter the details for the connection, click Save.

    To save the connection and immediately run the connection, click Run and Save.

    If needed, resolve any errors or missing information. After the connection creates successfully, the connection details display.

  2. To view details when the connection runs, click the Logs tab.
  3. To view an individual run log, expand the row table. For more information on resolving errors, see Troubleshooting.