Troubleshooting

Collect logs

Collect logs within Integrity Monitor

You must have either an Administrator or Content Administrator role in Integrity Monitor to collect logs.

To collect logs required for troubleshooting

  1. At the top right of the Integrity Monitor Home page, click Help .
  2. Click the Troubleshooting tab.
  3. Click Collect and then click Download to get the zipped file required by Tanium to assist you with troubleshooting.

Remove Integrity Monitor tools from endpoints

If needed, you can deploy a pre-configured package to remove Integrity Monitor tools from an endpoint or computer group.

Linux

  1. Using Interact or a Saved Question, run the Integrity Monitor Endpoint Tools Status Sensor and the Is Linux Sensor.
  2. From the Linux endpoints that have Integrity Monitors tools installed, drill down and select the targets.

  3. Deploy the Integrity Monitor - Remove Tools [Linux]  package to those targets.



Windows

  1. Using Interact or a saved question, run the Integrity Monitor Endpoint Tools Status Sensor and the Is Windows sensor.
  2. From the Windows endpoints that have Integrity Monitor tools installed, drill down and select the targets.
  3. Deploy the Integrity Monitor - Remove Tools [Windows]  package to those targets.

AIX

  1. Using Interact or a saved question, run the Integrity Monitor Endpoint Tools Status Sensor and the Is AIX sensor.
  2. From the Windows endpoints that have Integrity Monitor tools installed, drill down and select the targets.
  3. Deploy the Integrity Monitor Endpoint Tools Removal [AIX]  package to those targets.

Solaris

  1. Using Interact or a saved question, run the Integrity Monitor Endpoint Tools Status Sensor and the Is Solaris sensor.
  2. From the Windows endpoints that have Integrity Monitor tools installed, drill down and select the targets.
  3. Deploy the Integrity Monitor Endpoint Tools Removal [Solaris]  package to those targets.

Configure an interval between Integrity Monitor Endpoint Config actions

By default, all Deploy Integrity Monitor Endpoint Config actions are issued together hourly. For highly virtualized environments, endpoints with minimal hardware resources, or for installations with several monitors configured, you can stagger the Deploy Integrity Monitor Endpoint Config actions by configuring an interval between them.

  1. From the Integrity Monitor Home page, click Settings , and click the General Settings tab.
  2. In the Endpoint Config Distribution Interval section, enter the Distribution Interval Minutes to wait between issuing Deploy Integrity Monitor Endpoint Config actions.
  3. Redeploy monitors. For more information, see Deploy monitors.

Each action is issued the specified number of minutes after the previous action is issued. The time to run the entire sequence of actions is rounded up to the next hour to determine when the sequence starts again. For example, if there are 14 actions, and you specify 10 minutes for the interval, then it takes 2 hours and 20 minutes for all actions to run, and the sequence starts again at 3 hours from the time the first action was issued.

Distribute Integrity Monitor - Tools packages over time

By default, the Deploy Integrity Monitor - Tools [Linux] and Deploy Integrity Monitor - Tools [Windows] actions are issued to all targeted endpoints together hourly. To reduce network utilization, or to conserve hardware resources in highly virtualized environments, you can configure the Deploy Integrity Monitor - Tools actions to be distributed over time.

If you upgraded from a version earlier than 2.7, this setting is 15 minutes by default.

  1. From the Integrity Monitor Home page, click Settings , and click the General Settings tab.
  2. In the Tools Action Distribute Over Time section, enter a value for Distribute Over Time. This value specifies the number of minutes over which the Deploy Integrity Monitor - Tools action is randomly issued to all endpoints.

The interval for each Deploy Integrity Monitor - Tools action is automatically set to the sum of the command timeout for the action, the download timeout for the action, a 30-minute buffer for reissuing the action, a 5-minute buffer for action expiration, and the value set for Distribute Over Time. For example, if the command timeout is 20 minutes, the download timeout is 25 minutes, and the Distribute Over Time setting is 10 minutes, then the action interval is 1.5 hours (90 minutes).

Specify Diagnostic Settings

  1. To specify minimum log level, from the Integrity Monitor Home page, click Help .
  2. Click the Troubleshooting tab.
  3. Select the needed Log Level. The default for this setting is INFO.

Override Configuration Settings

While working with your TAM to troubleshoot an issue, you might need to upload a JSON file to override low-level configuration settings.




Updating these settings without careful consideration can cause serious system degradation. Do not override these settings unless you are working with your TAM.

These settings do not apply automatically to endpoints that use Client Recorder Extension 2.0. For more information about Client Recorder Extension and Integrity Monitor versions, see Upgrading to Integrity Monitor 2.0.

  1. From the Integrity Monitor Home page, click Help .
  2. Click the Troubleshooting tab.
  3. In the Configuration Override section, click Upload and browse to the JSON file.

  4. Click Import.

Manually install the Tanium Event Recorder Driver

The Tanium Event Recorder Driver is installed by default when you deploy a monitor that is configured to use it. For more information, see Create a new monitor. You can use the following procedure to assist in troubleshooting.

  1. From the Main menu, ask the question Get Tanium Driver Status from all machines and click Search.
  2. Select Install Recommended.
  3. From the Deploy Action page, select Install Tanium Driver.
  4. Validate successful installations by checking the validation query that runs at the end of the package installation.
  5. Collect the action logs from any endpoints that fail the validation query using Live Response.
  6. Run the action Remove Tanium Driver on any endpoints that return anything other than SERVICE_RUNNING for the Tanium Event Recorder Driver service status.

Monitor and troubleshoot Integrity Monitor Server Coverage

The following table lists contributing factors into why the Integrity Monitor Server Coverage metric might be lower than expected, and corrective actions you can make.

Contributing factor Corrective action
Endpoint health issues

Make sure that endpoints that are servers:

  • Have the minimum recommended free space. See Endpoints.
  • Do not report errors from the Integrity Monitor Endpoint Tools Status sensor.

    To identify endpoints with errors, you can ask the question Get Integrity Monitor Endpoint Tools Status matches "^Integrity Monitor is healthy$|^Integrity Monitor is not healthy$" from all machines.

Unaddressed installation failures Identify servers with Integrity Monitor installation failures, investigate the specific error codes, and address the conditions causing the failures.
The correlation engine (Integrity Monitor) process is not running

Make sure there are no conditions preventing the correlation engine process from running on servers included in the Integrity Monitor action group, such as inadequate security exclusions, errors installing Integrity Monitor tools, or issues that prevent scheduled actions from running.

Monitor and troubleshoot Mean Number of Unexpected Change Events per Endpoint or Expected vs Unexpected Change Events

The following table lists contributing factors into why the Mean Number of Unexpected Change Events per Endpoint metric might be higher than expected, or why the Expected vs Unexpected Change Events metric might show a high ratio of unexpected changes, and corrective actions you can make.

By default, the Expected vs Unexpected Change Events Trends panel displays a stacked area chart with the numbers of unlabeled and labeled events for each 24-hour period over time, which provides a visual representation of the ratio over time. To view the numeric ratio for a specific time period, you can view the panel as a donut chart.

  1. From the Effectiveness section of the Integrity Monitor Trends board, click Expected vs Unexpected Change Events to open the detailed view of the panel.
  2. For Chart Type, select donut chart .
  3. Select the Date Range for which you want to view the ratio of expected and unexpected events.
Contributing factor Corrective action
No integration with IT workflow Configure integration with ServiceNow Change Management for automatic labeling of events in approved change windows, and make sure that ServiceNow configuration items are properly mapped to Tanium endpoints. See Integrating with IT workflows.
Too few rules

Configure rules to appropriately label events that are performed by approved users, processes, and applications. See Working with rules.

Weak change controls that allow users to make unauthorized changes to systems

Work with the appropriate business units to define better change control policies and methods.

Uninstall Integrity Monitor

Consult with your TAM before you uninstall Integrity Monitor in a production environment so that you understand the potential repercussions.

  1. From the Main menu, click Solutions. Under Integrity Monitor, click Uninstall. Click Uninstall to complete the process.
  2. Delete any remaining Integrity Monitor related scheduled actions and action groups.
  3. Remove Integrity Monitor Tools from your endpoints. To see which endpoints have the Integrity Monitor tools installed, ask the question Get Integrity Monitor Tools Status from all machines. If any endpoints are returned by this question and you want to remove Integrity Monitor Tools from the endpoint, contact your TAM.
  4. The uninstall process creates a folder with a backup of the Integrity Monitor files called integrity-monitor-service-files. If you have access to the file system on the machine hosting the Module Server, you can keep or delete this folder. If any other Integrity Monitor artifacts remain on your Module Server, contact your TAM.

Uninstall Integrity Monitor tools from legacy Windows endpoints

Consult with your TAM before you uninstall Integrity Monitor from legacy Windows endpoints in a production environment so that you understand the potential repercussions.

If you used Integrity Monitor 2.2.1 or earlier with legacy Windows endpoints, and you want to remove the legacy Integrity Monitor tools from those endpoints, you can deploy the Integrity Monitor Remove Windows Legacy Tools package.

  1. In Interact, ask the question Get Is Windows from all machines with Windows OS Major Version matches ^5\..*|^6\.0. This question returns a single line that reports True in the Is Windows column, which represents all legacy Windows endpoints.
  2. Select the check box in the grid beside True.
  3. Click Deploy Action. Select the Integrity Monitor Remove Windows Legacy Tools package.
  4. Click Preview and then click Deploy Action.
  5. Confirm the action.