If Threat Response is not performing as expected, you might need to troubleshoot issues or change settings. For assistance, you can also contact your TAM.
Use Threat Response to compile a collection of logs relevant for troubleshooting.
- From the Threat Response home page, click Help , then click the Troubleshooting tab.
- Click Create Package.
- When the status of the package creation is complete, click Download Package.
The log zip file might take a few moments to appear in the download folder.
Collect troubleshooting information from endpoints
You can collect logs and other artifacts from individual endpoints to help resolve issues. Collecting logs from endpoints requires a live connection to the endpoint from which you want to gather troubleshooting information.
- From the Threat Response menu, click Live Endpoints.
- Create a live connection to the endpoint from which you want to collect troubleshooting information. For information on creating a live connection, see Connecting to live endpoints.
- Select the endpoint from which you want to collect troubleshooting information.
- Click Troubleshoot.
- Confirm that you want to collect troubleshooting information. A package is gathered from the endpoint and a link to the download package is displayed. The name of the link is the time stamp of the troubleshooting package. Click the link to download a ZIP file that contains troubleshooting information.
From the Main menu, click Management > System Notifications. These notifications show non-match alerts, including when alert throttling is enabled on any endpoints. To delete a notification, select the row and click Delete.
You can view the status of and review other historic data for Threat Response tasks.
From the Threat Response menu, click Management > Tasks.
For each task that you have initiated, you can view the status, the start, stop, and creation time, and the task ID. The types of Threat Response tasks for which you can view this detailed information include:
- Deploy Profile
- Remove Profile
- Deploy Intel
- Deploy Tools
- Start Index
- Group Migration
- Content Migration
- Suppress Alerts
- Response Action
In the event of an error, you can locate a specific task and expand the cell to view results and metadata from log files. The data that displays in the advanced details view provides information that is useful for troubleshooting and saves the time of trying to locate debugging information in lengthy log files or in formats that are not intended to be human readable.
Ask the question: Get Threat Response Status from all machines.
The results returned are divided by operating system, component versions, and the endpoints that do not have the tools installed. The Threat Response tools status is evaluated when the Threat Response service restarts or is updated.
For more information, see Tanium Interact User Guide: Asking questions.
Adjust the throttling settings to control how many alerts you are getting on the endpoint. Both endpoint and service alert throttling are enabled by default.
Configure throttling for Signal alerts on endpoints
You can configure throttling of Signal alerts on the endpoint when you create an engine configuration. By default, Signal alert throttling on an endpoint is enabled and occurs when five events on a single piece of intel occur within five minutes.
- Go to Management > Engine. Select a configuration.
- In the Advanced Engine Settings section, update the settings for Signal throttling.
Save the engine configuration, and deploy any profiles that use that configuration for the endpoint throttling to be deployed.
- If Signal alert throttling occurs, notifications about the event and the endpoint that has throttling enabled display on the System Notifications page.
Configure match alert throttling
You can configure match alert throttling for the engine. This service-level throttling can apply for quick scan alerts and all types of intel. By default, match alert throttling is enabled and occurs when 100 events on a single piece of intel occur within 20 minutes.
- From the Threat Response home page, click Settings then click the Service tab. Click Intel.
- Edit the settings for match throttling. You can adjust when throttling occurs, and adjust the cooloff period, which controls how long alerts continue to be throttled.
- Click Save.
- If alert throttling occurs, you get notifications that throttling is enabled display on the System Notifications page.
You can change most engine settings in the advanced settings of an engine configuration. You might need to manually run scripts from the Tanium Client directory on the endpoint to edit the engine configuration.
- get-config provides all of the available options and their current values.
- set-config changes the value or clears it, returning to the default behavior.
You can also find more information about engine configuration options in the engine documentation.
If Linux endpoint events are not being recorded, they might be missing the audit daemon and audispd services. Ideally, the audit daemon is installed and configured before installing the Threat Response module, but it is possible for endpoints to come online at a later time.
- (Optional) Create the auditd package.
You can either create a general installation package and put the logic in the scripts or you can have a simple script and put the logic in the Tanium query. See Tanium Core Platform User Guide: Creating and managing packages.
- Ask the question: Get Installed Application Exists[audit] from all machines with Is Linux containing "true".
- Deploy the appropriate auditd package to the identified endpoints.
If you need to distribute the package to a large number of endpoints, spread the changes out over time to avoid a negative impact on the network.
If you are having difficulty making a live connection to an endpoint, diagnose the issue with this workflow.
A configuration that has the recorder enabled is required to make a live connection.
- Ask the Get Threat Response - Status saved question and verify that the endpoint status is No Issues found.
If the status is Install Needed, run the Distribute Tanium Trace Tools action.
- Ask the Get Trace Endpoint Certificate Installed saved question and verify that the endpoint status is True.
Run the Trace - Install Endpoint Certificate action from the Tanium Threat Response action group action if necessary.
- Click Settings , then click the Service tab. Select Misc. Verify that the Module Server IP address is correct.
- Verify that a firewall is not blocking the connection from the endpoints to the Module Server:
- From a remote computer, browse to https://<module server IP address or FQDN>:17444/status.
- If you get a message that the site cannot be reached, update firewall rules.
- Verify that you are using the correct endpoint identification to initiate the connection.
If initiating a connection from Threat Response, you must type in the IP address or Computer Name, usually FQDN, that is recognized by Tanium or the connection fails. Alternatively, you can:
- Use Interact to ask the Get Computer Name and IP Address from all machines question.
- From the results grid, select an endpoint and deploy the Start Tanium Trace Session package.
- Review the Start Tanium Trace Session action to verify that it has completed.
- Access the endpoint and review the Trace Websocket Client file.
Operating system File path
<tanium_client_directory>\Tools\Trace\TraceWebSocketClient.log Linux <tanium_client_directory>/Tools/Trace/TraceWebsocketClient.log Mac <tanium_client_directory>/Tools/Trace/TraceWebsocketClient.log
If needed, increase the log level to debug:
- Create an .ini file with the same name and location as the websocket client location.
For example, <tanium_client_directory>\Tools\Trace\TraceWebsocketClient.ini.
- Add the line: logging.loggers.root.level=debug.
- Try to connect again and review the results in the log file.
- Create an .ini file with the same name and location as the websocket client location.
- If you cannot resolve the problem, contact your TAM with the following information:
- Trace service logs
- Action logs
- Trace Zone Proxy and Hub logs, if available
- Trace Websocket Client log
For configurations with the Trace Zone proxy service, see Troubleshoot the Trace zone proxy service for more information.
When Threat Response is installed, the installation process fills in the Module Server IP address that the endpoints use to connect. If this address changes, you might need to update the Service Settings.
- From the Threat Response home page, click Settings , then click the Service tab. Click Misc.
- Enter the IP address of the Module Server.
- Click Save.
Certificates authorize live connections between Threat Response and endpoints. The certificate must be in PEM format. In the certificate signing request, enable both web server and web client authentication.
As certificates near expiration, a warning message is displayed.
- From the Threat Response home page, click Settings , then click the Certificate tab.
- Select an option:
- Click Generate self-signed certificate.
- Upload a certificate and private key.
- Click Install.
You might need to manually start or stop the recorder. The recorder does not restart automatically.
For example, if the database size or CPU usage limits are exceeded, the recorder is automatically stopped. Resolve the underlying issue and restart the recorder. Or, if you find that the recorder is using more system resources than expected, you can stop the recorder and troubleshoot the issue.
- Identify the computer groups that contain endpoints on which you want to stop the recorder.
- Deploy a profile that does not contain a recorder configuration to the computer groups you want to target.
- You can optionally create a live endpoint connection to specific endpoints to troubleshoot any issues.
To start the recorder, deploy a profile that has a recorder configuration to the targeted computer groups.
If the database is identified as corrupt, you might need to clear the endpoint database.
- Use a question to target the affected endpoints.
For example, ask Get Trace Invalid File Operations from all machines.
- Drill down to the endpoints that return true.
- Deploy the Trace - Recreate Database [OS] or Threat Response - Recreate Database [OS] package as an action.
For more information, see the Tanium Core Platform User Guide: Managing and creating Packages or the Tanium Interact User Guide: Using Deploy Action.
You might need to remove Trace from the Tanium Module Server for troubleshooting purposes.
- From the Tanium Console, click Solutions.
- Locate Threat Response, and then click Uninstall.
- To confirm, return to the Solutions page and check that the Import button is available.
Last updated: 11/7/2019 11:46 AM | Feedback