Troubleshooting

Collect logs

Collect server logs within Integrity Monitor

Server logs might be required by Tanium to assist you with troubleshooting.

You must have either an Administrator or Content Administrator role in Integrity Monitor to collect server logs.

  1. From the Integrity Monitor Overview page, click Help .
  2. Click the Troubleshooting tab.
  3. Click Collect and then click Download to download a ZIP file containing the logs.

Change the logging level

  1. To specify minimum log level, from the Integrity Monitor Overview page, click Help .
  2. Click the Troubleshooting tab.
  3. Select a Log Level. The default setting is INFO.

This update changes the log level for future logging. It does not affect the data that is available in the support package for previously logged events.

Generate endpoint debugging logs

Endpoint debugging logs might be required by Tanium to assist you with troubleshooting.

  1. From the Main menu, go to Modules > Interact.
  2. Ask a question to target the endpoints for which you want to generate debugging logs.
  3. Select the endpoints for which you want to generate debugging logs.
  4. Click Deploy Action.
  5. On the Deploy Action page, enter Set Windows Tanium Client Logging Level, Set Linux Tanium Client Logging Level, or Set Mac Tanium Client Logging Level in the Enter package name here box and select the appropriate action for the OS of the endpoints you are targeting.
  6. In the Log Level box, enter 41.
  7. Click Show preview to continue.
  8. A results grid displays at the bottom of the page showing you the targeted endpoints for your action. If you are satisfied with the results, click Deploy Action.
  9. Click Go back to return to question results, and reselect the endpoints for which you want to generate debugging logs.
  10. Click Deploy Action.
  11. On the Deploy Action page, enter Integrity Monitor Endpoint Debug Zip in the Enter package name here box, and select Integrity Monitor Endpoint Debug Zip [Windows] or Integrity Monitor Endpoint Debug Zip [Linux] as appropriate for the endpoints you are targeting.
  12. Click Show preview to continue.
  13. A results grid displays at the bottom of the page showing you the targeted endpoints for your action. If you are satisfied with the results, click Deploy Action.

    Each targeted endpoint generates a debug log located at <Tanium Client>\Tools\IM\debug\tanium_im_debug.zip (for Windows) or <Tanium Client>/Tools/IM/debug/tanium_im_debug.zip (for non-Windows).

Remove endpoint debugging logs from endpoints

(Optional) After you obtain endpoint debugging logs, you can remove the logs from endpoints.

  1. From the Main menu, go to Modules > Interact.
  2. Ask a question to target the endpoints from which you want to remove debugging logs.
  3. Select the endpoints from which you want to remove debugging logs.
  4. Click Deploy Action.
  5. On the Deploy Action page, enter Integrity Monitor Endpoint Delete Debug Zip in the Enter package name here box, and select Integrity Monitor Endpoint Delete Debug Zip [Windows] or Integrity Monitor Endpoint Delete Debug Zip [Linux] as appropriate for the endpoints you are targeting.
  6. Click Show preview to continue.
  7. A results grid displays at the bottom of the page showing you the targeted endpoints for your action. If you are satisfied with the results, click Deploy Action.

    The tanium_im_debug.zip file is removed from the targeted endpoints.

Manually install the Tanium Event Recorder Driver

The Tanium Event Recorder Driver is installed by default when you deploy a monitor that is configured to use it. For more information, see Create a new monitor. You can use the following procedure to assist in troubleshooting.

  1. From the Main menu, ask the question Get Tanium Driver Status from all machines and click Search.
  2. Select Install Recommended.
  3. From the Deploy Action page, select Install Tanium Driver.
  4. Validate successful installations by checking the validation query that runs at the end of the package installation.
  5. Collect the action logs from any endpoints that fail the validation query using Live Response.
  6. Run the action Remove Tanium Driver on any endpoints that return anything other than SERVICE_RUNNING for the Tanium Event Recorder Driver service status.

File, network, or security events are not displayed on Oracle Enterprise Linux server

If you are not seeing file, network, or security events in the recorder results, you can disable SELinux. When SELinux is enabled and the auditd fallback is disabled on Oracle Enterprise Linux, only process information is returned. Alternatively, ensure that the Client Recorder Extension configuration parameters are set as follows:

  • CX.recorder.AuditdStopAuditdService is set to 0.
  • CX.recorder.AuditdEnableAudispdFallback is set to 1.

For more information, see Client Recorder Extension User Guide: Configuring recorded events .

Monitor and troubleshoot Integrity Monitor Server Coverage

The following table lists contributing factors into why the Integrity Monitor Server Coverage metric might be lower than expected, and corrective actions you can make. To get detailed health information about endpoints that are included in the Initializing and Needs Attention categories, ask the question: Get Integrity Monitor Endpoint Tools Status from all machines with Integrity Monitor - Coverage Status matches ^Needs\ Attention|^Initializing.

Contributing factor Corrective action
Integrity Monitor tools are initializing The Initializing category includes endpoints on which tools are still initializing. If tools were recently deployed, wait for initialization to complete.
Endpoint health issues

Make sure that endpoints that are servers:

  • Have the minimum recommended free space. See Endpoints.
  • Do not report errors from the Integrity Monitor Endpoint Tools Status sensor.

    To identify endpoints with errors, you can ask the question: Get Integrity Monitor Endpoint Tools Status matches "^Integrity Monitor is healthy$|^Integrity Monitor is not healthy$" from all machines.

Unaddressed installation failures Identify servers with Integrity Monitor installation failures, investigate the specific error codes, and address the conditions that are causing the failures.
The correlation engine (Integrity Monitor) process is not running

Make sure no conditions are preventing the correlation engine process from running on servers in the Integrity Monitor action group. These conditions might include inadequate security exclusions, errors installing Integrity Monitor tools, or issues that prevent scheduled actions from running.

Monitor and troubleshoot Mean Unexpected Change Events per Endpoint or Expected vs Unexpected Change Events

The following table lists contributing factors into why the Mean Unexpected Change Events per Endpoint metric might be higher than expected, or why the Expected vs Unexpected Change Events metric might show a high ratio of unexpected changes, and corrective actions you can make.

By default, the Expected vs Unexpected Change Events Trends panel displays a stacked area chart with the numbers of unlabeled and labeled events for each 24-hour period over time, which provides a visual representation of the ratio over time. To view the numeric ratio for a specific time period, you can view the panel as a donut chart.

  1. From the Effectiveness section of the Integrity Monitor Trends board, click Expected vs Unexpected Change Events to open the detailed view of the panel.
  2. For Chart Type, select Donut chart .
  3. Select the Date Range for which you want to view the ratio of expected and unexpected events.
Contributing factor Corrective action
No integration with IT workflow Configure integration with ServiceNow Change Management for automatic labeling of events in approved change windows, and make sure that ServiceNow configuration items are properly mapped to Tanium endpoints. See Integrating with IT workflows.
Too few rules

Configure rules to appropriately label events that are performed by approved users, processes, and applications. See Labeling events.

Weak change controls that allow users to make unauthorized changes to systems

Work with the appropriate business units to define better change control policies and methods.

Uninstall Integrity Monitor

Contact Tanium Support before you uninstall Integrity Monitor in a production environment so that you understand the potential repercussions.

  1. From the Main menu, go to Administration > Configuration > Solutions.
  2. Select the check box for Integrity Monitor, and click Delete Selected . Click Uninstall to complete the process.
  3. Delete any remaining Integrity Monitor related scheduled actions and action groups.
  4. Remove Integrity Monitor tools from your endpoints. For more information, see Remove Integrity Monitor tools from endpoints.
  5. The uninstall process creates a folder with a backup of the Integrity Monitor files called integrity-monitor-service-files. If you have access to the file system on the machine hosting the Module Server, you can keep or delete this folder. If any other Integrity Monitor artifacts remain on your Module Server, Contact Tanium Support.

Remove Integrity Monitor tools from endpoints

You can deploy an action to remove Integrity Monitor tools from an endpoint or computer group. Separate actions are available for Windows and non-Windows endpoints.

  1. In Interact, ask the question: Get Endpoint Configuration - Tools Status from all machines with Is Windows equals True or Get Endpoint Configuration - Endpoint Tools Status from all machines with Is Windows equals False.
  2. In the results, select the row for Integrity Monitor, drill down as necessary, and select the targets from which you want to remove Integrity Monitor tools. For more information, see Tanium Interact User Guide: Managing question results.
  3. Click Deploy Action.
  4. On the Deploy Action page, enter Endpoint Configuration - Uninstall in the Enter package name here box, and select Endpoint Configuration - Uninstall Tool [Windows] or Endpoint Configuration - Uninstall Tool [Non-Windows], depending on the endpoints you are targeting.
  5. For Tool Name, select Integrity Monitor.

  6. (Optional) To allow automatic reinstallation of the tools when monitors are redeployed, clear the selection for Block reinstallation.

    If reinstallation is not blocked on an endpoint that is still targeted by a monitor, tools are automatically reinstalled on that endpoint almost immediately. If reinstallation is blocked on an endpoint, you must deploy the Endpoint Configuration - Unblock Tool [Windows] or Endpoint Configuration - Unblock Tool [Non-Windows] package (depending on the targeted endpoints) before the tools can be reinstalled during monitor deployment.

  7. (Optional) To remove all Integrity Monitor databases and logs from the endpoints, clear the selection for Soft uninstall.

  8. (Optional) To also remove any tools that were dependencies of the Integrity Monitor tools (such as Recorder and Index) that are not dependencies for tools from other modules, select Remove unreferenced dependencies.

  9. Click Show preview to continue.
  10. A results grid displays at the bottom of the page showing you the targeted endpoints for your action. If you are satisfied with the results, click Deploy Action.

If you have enabled Endpoint Configuration, tool removal must be approved in Endpoint Configuration before tools are removed from endpoints.

Resolve issues with legacy Client Recorder Extension installations

If Tanium Endpoint Configuration detects endpoints that have legacy versions of the Client Recorder Extension installed, it reports the endpoint as Unsupported in the recorder column of the results grid when you ask the question: Get Endpoint Configuration - Tools Status. If Client Recorder Extension version 1.x exists on a targeted endpoint, you must remove it before you install Client Recorder Extension version 2.x tools. To target endpoints where Client Recorder Extension version 1.x exists, ask the question: Legacy - Recorder Installed. If the Supported Endpoints column displays No, you must remove Client Recorder Extension version 1.x from the endpoint before you install Client Recorder Extension 2.x tools. To remove Client Recorder Extension version 1.x, deploy the Recorder - Remove Legacy Recorder [Operating System] package to targeted endpoints.

Contact Tanium Support

To contact Tanium Support for help, send an email to [email protected].