Configuring Elasticsearch destinations

With Elasticsearch, you can search, analyze, and get actionable insights in real time from almost any type of structured and unstructured data source. With Connect, Tanium can write data directly to Elasticsearch.

Specify general connection information

1. On the Connect Overview page, scroll to the Connections section and click Create Connection.
2. Enter a name and description for the connection.
3. (Optional) In the General Information section, expand Advanced to configure the following settings:

   Log Level

   By default, Log Level is set to Information. To reduce the amount of logging, you can set Log Level to Warning, Error, or Fatal.
   
   
   

   Override Log Level

   If you are debugging the connection, select Override Log Level to set a Temporary Log Level (such as Trace or Debug) on this connection for a selected Number of Runs (up to 24). A scheduled or manual connection run, once started, counts towards the number of runs, regardless of the connection status. After the number of runs elapse, the logging for this connection returns to the Log Level you selected to prevent finer-grained logging from consuming additional resources for an indefinite number of runs.

   Minimum Pass Percentage

   Minimum percentage of the expected rows that must be processed for the connection to succeed.

   Memory Ceiling (GB)

   Maximum memory for the node process to run the connection. This defaults to 1 GB per connection, and cannot exceed the global maximum sum of memory for all running connections (by default, 8 GB). Increase this setting if a connection frequently exhibits out of memory errors while running.

   If the sum of simultaneously scheduled connection Memory Ceiling values exceed the global Memory Ceiling, connections run until the global Memory Ceiling is reached, then any remaining connections enter a waiting queue if you select the Queue Connections configuration setting, or fail if you clear the Queue Connections configuration setting.

Configure the connection source

The connection source determines what data you are sending to the destination. This data is usually information from Tanium, such as a saved question, question log, client status, or event. The settings vary depending on which source you choose.

Connection sources

Action History

The action history is a record of all actions issued by console operators. To view this record in Tanium, go to Administration > Actions > Action History. For more information, see Tanium Console User Guide: Manage actions that are completed or in progress.

Client Status

Client Status, previously named System Status, includes the state of all the endpoints, including some useful information about the endpoint like IP Address, position in the network, and the last time it registered with the Tanium Server. For more information about the Client Status data, see Tanium Console User Guide: View the status of Tanium Client registration and communication.

Event

Tanium solutions, like Tanium™ Discover, Tanium™ Integrity Monitor, and Tanium™ Threat Response, can forward events to Connect as a data source. These events can then be used as a connection source in a connection and sent to any of the available connection destinations. For more information, see Tanium Discover User Guide: Configure event notifications, Tanium Integrity Monitor User Guide: Sending events from basic monitors, and Tanium Threat Response User Guide: Exporting audit data.

Question History

The question history log is a history of every question that has been asked. When you are using the question log as a data source in Connect, you can filter the log in several ways to reduce the total volume of data being sent. For more information, see Tanium Console User Guide: Question history.

Saved Question

A saved question is a Tanium question that you want to ask on a repeated basis. For more information about saved questions, see Tanium Interact User Guide: Managing saved questions.

When you select a saved question as a source, the Computer Group drop-down defaults to No Filter, which does not filter the saved question with a computer group. Select No Filter if you want to send recent saved question results from all endpoints to the destination. The endpoint results are subject to the computer group management rights of the user configuring the connection, and might not match the endpoint membership of the All Computers computer group. Select the All Computers computer group if you want to explicitly filter the saved question on the All Computers computer group.

You can use the following settings for saved question sources:

Setting	Description
Flatten Results	You might want to enable the Flatten Results setting to process results as individual records. For example, you might want to get notified when you see a new MD5 hash on a machine. Without the Flatten Results setting enabled, the entire data set that is retrieved by the saved question from a machine, such as all MD5 hashes, is considered to be a single record. Any change that is made to this data set shows up in the destination. By enabling the Flatten Results setting, Connect processes the new hashes on an individual basis (one MD5 hash from one machine) instead of all hashes from a machine as a single record.
Hide Errors	If the saved question returns an error, you can use the Hide Errors setting to prevent the error results from getting sent to the destination.
Hide No Results	If the saved question returns [No results], you can use the Hide No Results setting to prevent this result from being sent to the destination.
Include Recent Answers	If you want to include results from machines that are offline, select Include Recent Answers, which returns the most recent answer to the saved question for the offline endpoint. If you select Include Recent Answers, for the best results, you should also select No Filter from the Computer Group drop-down.
Answer Complete Percent	Results are returned when the saved question returns the configured complete percent value. Any results that come in after the configured percent value has passed are not sent to the destination. If you are finding that the data returned from the saved question is incomplete in your destination, you can disable this setting by setting it to 0. If disabled, all data is returned after the timeout passes.
Timeout	Minutes to wait for clients to reply before returning processed results when Answer Complete Percent is set to 0. If the Answer Complete Percent value is not met at the end of the time limit, then the connection run is marked as a failure. The maximum timeout is 10 minutes.
Batchsize	Number of rows that are returned for the saved question results at one time. This setting might vary depending on your destination.

Server Information Source

Use the server information in the following location as a connection source: https://<tanium_server>/info.json.

Tanium™ Asset

Tanium Asset comes with a set of predefined reports to help you prepare for audit and inventory activities. You can also create your own custom reports and views. For each report or view, you can create a connection that specifies a report or view as a data source. Currently supported destinations include Email, File, HTTP, Socket Receiver, Splunk, and SQL Server. For more information, see Tanium Asset User Guide: Asset overview.

Tanium™ Audit Source

Tanium Server keeps detailed audit logs for server configuration and settings changes. However, accessing these logs requires direct access to the Tanium database. To access the audit logs, you can set them up as a data source in Connect. For more information, see Tanium Security Recommendations Guide: Enable and forward Tanium logs.

For information on data available with the Tanium Audit Source, see Reference: Tanium Audit Source data.

Tanium™ Comply

Tanium Comply enables you to export compliance and vulnerability findings to help support enterprise compliance goals. Use the Tanium Comply (Findings) source to export all compliance and all vulnerability findings. Use the Tanium Comply (Assessments) source to export all vulnerability assessments. For more information, see Tanium Comply User Guide: Exporting findings and assessments.

Tanium™ Data Service

The Tanium Data Service enables you to see stored sensor results for endpoints that are offline at the moment you issue a saved question. For more information, see Tanium Console User Guide: Manage sensor results collection.

Tanium™ Direct Connect

Tanium Direct Connect enables other Tanium modules to establish sessions with endpoints. You can create a connection that generates an audit report of Direct Connect sessions and actions that users performed on endpoints during Direct Connect sessions. For more information, see Tanium Direct Connect User Guide: Exporting an audit log.

Tanium™ Discover

Tanium Discover contains reports that maintain an inventory of interfaces in your environment. For each report, you can create a connection that specifies a report as a data source. For more information, see Tanium Discover User Guide: Discover overview.

Tanium™ Endpoint Configuration

Tanium Endpoint Configuration enables you to deliver configuration information to endpoints consistently for all Tanium solutions that are available in an environment. You can create a connection that generates an audit report of all Endpoint Configuration management actions, manifest actions, and configuration changes. For more information, see Tanium Endpoint Configuration User Guide: Exporting an audit log.

Tanium™ Impact

Tanium Impact identifies the users, groups, and endpoints that have the highest potential impact in your organization if compromised, based on the impact rating. You can create a connection that specifies all users, groups, or endpoints with the highest impact as a data source. For more information, see Tanium Impact User Guide: Impact overview.

Tanium™ Integrity Monitor

Tanium Integrity Monitor enables you to define watchlists of files, directories, and Windows registry paths that you want to monitor for changes. Use the Tanium Integrity Monitor source to export watchlist data. For more information, see Tanium Integrity Monitor User Guide: Integrity Monitor overview.

Tanium™ Reporting

Use Tanium Reporting to create custom reports and dashboards from data that is collected by the Tanium Data Service. You can create a connection to export report data or a dashboard. For more information, see Tanium Reporting User Guide: Export reports through Tanium Connect and Tanium Reporting User Guide: Export dashboards through Tanium Connect.

Tanium™ Reputation

Tanium Reputation is an aggregated repository of reputation data from various sources, including Palo Alto Networks WildFire, ReversingLabs, and VirusTotal. You can choose which type of status to include, such as only malicious or suspicious content. You can choose to include the full report, which includes the detailed information from the reputation source, not just the status of the reputation item. You must have one or more reputation sources configured to get information from this connection source. For more information, see Tanium Reputation User Guide: Reputation overview.

Tanium™ Threat Response

Tanium Threat Response contains audit reports for actions that were performed in Threat Response. For each report, you can create a connection that specifies a report as a data source. For more information, see Tanium Threat Response User Guide: Threat Response overview.

Tanium™ Trends

Tanium Trends provides data visualization panels from saved question or module sources. You can create boards that organize one or more panels. For each board, you can create a connection that specifies a board as a data source in HTML format. Valid destinations are AWS S3, Email, or File. For more information, see Tanium Trends User Guide: Trends overview.

Configure Elasticsearch destination

1. In the Configuration section, provide a name for the destination.

   • Specify a unique name to save the configuration information as a new destination. Select New, and then enter a Destination Name.

   • Select an existing destination. Select Existing, and then select a destination from the Destination Name drop-down list. If you edit the settings for an existing destination, all connections that use that destination are affected.

   • Copy an existing destination. Select New, and then click Copy Settings. Select a destination from the drop-down list, click Apply, and update the Destination Name.

2. Define a URL to access the Elasticsearch API.
   You can use a variable for the URL, as listed in Reference: Variables. The URL is in the following format:
   <protocol>://<elasticSearchHost>:<elasticSearchPort>/<index>/_bulk
   where:
   
   

   protocol

   Specifies the protocol for accessing the Elasticsearch server (http or https). If you specify https for the URL, tThe connection validates the Transport Layer Security (TLS) certificate. You can also select Trust on First Use to accept the certificate presented from the server and trust only that certificate for future connection runs.

   elasticSearchHost

   Specifies the fully qualified domain name (FQDN) or IP of the Elasticsearch server.

   elasticSearchPort

   Specifies the port on which the Elasticsearch server is listening. The default is: 9200.

   index

   Specifies the unique identifier for this API request. The common format for this identifier is: {name}-{0:yyyy.MM.dd}. This date format creates a unique index each day.

   If you submit an external access request for traffic, your configured destination FQDN or IP address, port, and protocol must match the FQDN or IP address, port, and protocol submitted in the external access request.

3. In the Advanced section for the destination, select Use Authentication if you want to use authentication with Elasticsearch, then enter a User Name and Password.

   If you specify http for the URL, authentication information is encrypted at rest, but passed unencrypted to the destination.

4. In the Advanced section, define the Batch Size, which determines the number of messages to batch before sending data to Elasticsearch, and the connection Timeout in seconds.
5. If you have a proxy configured for your Tanium Module Server, select Use Tanium Module Server Proxy Setting. When selected, the proxy makes the outbound connection to the specified Elasticsearch instance provided in the URL.

Configure filters

(Optional) In the Configure Output > Filters section, you can specify filters to modify the data that you are getting from your connection source before it is sent to the destination.

For more information about the types of filters you can configure, see Reference: Filtering options.

Format data for Elasticsearch

In the Configure Output > Format > Advanced section, choose the columns that you want to use as an _id metadata value in Elasticsearch. The _id value is a hash value generated from the combination of the selected column values, before applying any configured Value Escape Regex regular expressions and Value Escape Replace value substitutions. If you select at least one column, the connection data includes an additional column (id) which also contains the _id hash value. If you do not select any columns, Elasticsearch generates a unique _id value.

In the Configure Output > Format > Advanced section, choose the Column names for hashed _id metadata value used in Elasticsearch. The _id value is a hash value generated from the combination of the selected column values, before applying any configured Value Escape Regex regular expressions and Value Escape Replace value substitutions. If you select at least one column, the connection data includes an additional column (id) which also contains the _id hash value. If you do not select any columns, Elasticsearch generates a unique _id value.

Select columns whose values generally remain static, such as ComputerId, ComputerName, or ComputerSerialNumber.

If a saved question is selected as a source, select Group columns by sensor to return the results as an array of objects instead of an array of simple value types. You cannot customize the columns with this option.

In the Configure Output > Columns section, you can change the Destination Label of each column and Value Type to force the column to be a String, Numeric, or Date/Time value.

If you choose Numeric for the value, you can specify a default value that is used if the data cannot be coerced into a numeric value. You can specify any negative or positive number.

If you choose Date/Time for the value, specify the format that you want to use for the column. For more information about using a variable, see Time stamp variables.

For more information about column customizations, see Reference: Column customizations.

Schedule the connection

Connections can run at a highly configurable time interval, such as multiple times per hour, day, week, or month.

If you do not enable the schedule, the connection only runs when you manually run it, unless you configure an Event source. Connections with Event sources only run when a configured event is detected, and cannot be scheduled or manually run.

Use the Schedule section to update the schedule:

• Select Enable schedule.
• In the Schedule Type, select Basic to build a schedule with the provided controls.
  
• To view or edit the Cron expression directly, select Advanced - Define as a Cron Expression, and use the Advanced field to edit the Cron expression.
  
  
  
  
  

For more information about Cron syntax, see Reference: Cron syntax.

Save and verify connection

1. After you enter the details for the connection, click Save.

   To save the connection and immediately run the connection, click Run and Save.

   If needed, resolve any errors or missing information. After the connection creates successfully, the connection details display.

2. To view details when the connection runs, click the Logs tab.
3. To view an individual run log, expand the row table. For more information on resolving errors, see Troubleshooting.