Troubleshooting

Basic troubleshooting tips

  • TPAN report: Regularly run and review the the Tanium Platform Analyzer (TPAN) report. The report can facilitate future troubleshooting regardless of whether your deployment currently has issues. For details about the best practices for TPAN settings, see Tanium Core Platform Deployment Reference Guide: Troubleshooting workflow.TPAN report
  • Solution versions: Installing later versions of Tanium solutions (modules and shared services) might resolve unexpected behavior. The Tanium Console version appears in the Console header. To check the currently installed versions of other solutions, and to update them if necessary, see Managing Tanium solutions.

    Install the most recent version of every solution.

  • Local Error Log: Review any error messages in the Tanium Console error log. See Work with the Console error log.
  • Other logs: The following sections describe troubleshooting tasks that you perform through the Tanium Console or other Tanium solutions. If these tasks are insufficient for diagnosing issues, you can view logs on the Tanium Core Platform servers. For details about optimizing the logging level for troubleshooting, see Tanium Core Platform Deployment Reference Guide: Logging levels.
You can view the Local Error Log through the Tanium Console, but the requirements for viewing other log types vary by infrastructure. See Tanium Core Platform Deployment Reference Guide: Log and report types.

If the browser that you use to access the Tanium Console displays an HTTP response status code in the 500 to 599 range, this indicates a server error. However, the responsible server might or might not be a Tanium Core Platform server. See Troubleshoot core platform server issues.

Troubleshoot Console issues

Troubleshoot Console access

If you cannot access the Tanium Console, review the following causes and troubleshooting tasks.

If you can access the Console but not a particular Tanium solution or Console feature, the issue might be related to user permissions. See Troubleshoot role-based access control (RBAC) issues.

Console is unavailable

If you cannot reach the Tanium Console through your browser, review the following issues and remediation tasks:

Console authentication errors

If you cannot sign in to the Tanium Console, review the following issues and remediation tasks:

  • User configuration: Ask an administrator who has permissions for managing users to verify that your account exists in Administration > Permissions > Users. For the steps to view or create accounts, see Managing users.

  • User account status: Ask your IdP administrator to verify that the account is not renamed, deleted, disabled, or locked out in the identity store.

If you can access the Console but not a particular Tanium solution or Console feature, the issue might be related to user permissions. See Troubleshoot role-based access control (RBAC) issues.

  • User configuration: Ask an administrator who has permissions for managing users to verify that your account exists in Administration > Permissions > Users. For the steps to view or create accounts, see Managing users.

  • User account status: Verify that your account is not renamed, deleted, disabled, or locked out. The specific steps depend on the type of authentication that is configured for your account:
    • LDAP/AD: If your account is imported, ask your Active Directory (AD) or Lightweight Directory Access Protocol (LDAP) administrator to check the account status. You can also review the LDAP log. See Tanium Core Platform Deployment Reference Guide: LDAP log.
    • SAML: For accounts that authenticate through Security Assertion Markup Language (SAML), ask your IdP administrator to check the account status in the identity store.
    • Local authentication (Tanium Appliance only): If your account uses the local authentication service, see Tanium Appliance Deployment Guide: Configure the local authentication service.
    • Windows user database (Windows deployment only) For accounts that are defined locally on a Tanium Server, check the account status in the Windows user database: From the Windows Windows menu, open the Control Panel, click User Accounts, and click Manage another account.
  • Smart Card access: If you configured Console access through smart cards, see Tanium Core Platform Deployment Reference Guide: Troubleshoot smart card authentication.
  • Logs: Review the history of authentication events in the authentication log. See Tanium Core Platform Deployment Reference Guide: Authentication log.
  • Tanium Server upgrades: In rare cases, if the Tanium Server was upgraded since your last sign-in session, you might have to clear your browser cache or try a different browser.

Troubleshoot unexpected Console behavior

If the Tanium Console behaves in unexpected ways, verify that the latest Console version is installed. If a version update is available, import it. To verify the version and import an update, see Import Console UI updates.

Troubleshoot session termination issues

If Tanium Console user sessions terminate prematurely, review the events and platform settings that affect the duration of a user session. See:

The browser that you use to access the Console might display the 401 Unauthorized error when a session expires.

Troubleshoot license issues

To troubleshoot issues with the Tanium license, see Troubleshoot license issues.

Work with the Console error log

The Tanium Console maintains an error log on the local host computer for your web browser. It includes details on the last 100 errors that were returned to the Console in response to actions that you performed through the browser. For example, the log records errors that are associated with attempting to save a configuration or import a content file. The Console maintains a separate log for each browser that you use.

  1. In the Main menu, click User and select Local Error Log.
  2. (Optional) Expand Expand a log entry and click Copy Copy to copy the log details to the clipboard.
  3. (Optional) Click Clear Log if you want to remove all the log entries. For example, you might want to remove the entries after you finish resolving the associated issues.

Troubleshoot Tanium Cloud issues

In some cases, unexpected behaviors that manifest in the Tanium Console might result from issues in your Tanium™ Cloud instance. The Tanium Cloud Management Portal (CMP) indicates errors or warnings on the Administration page. For details, see Tanium Cloud Deployment Guide: Troubleshooting Tanium Cloud.

Troubleshoot core platform server issues

In some cases, unexpected behaviors that manifest in the Tanium Console might result from issues in Tanium Core Platform servers. The following sections describe server troubleshooting tasks that you can perform through the Console.

You can perform additional troubleshooting and remediation tasks on the servers through the command-line interface (CLI) or other means. For example, you can restart services, upgrade servers, or view installation logs. For details, see:

The Solutions page indicates whether server updates are available. See Tanium solutions overview.

To troubleshoot proxy server issues, test the proxy settings and update them as necessary. See Configure proxy server settings.

If the browser that you use to access the Console displays an HTTP response status code in the 500 to 599 range, this indicates a server error. The responsible server might or might not be a Tanium Core Platform server.

Troubleshoot platform server trust issues

In an active-active deployment, Tanium Servers must trust each other for synchronization. In a deployment with Tanium Zone Servers, trust is also required between Tanium Servers and Zone Server Hubs, and each Zone Server must be mapped to a hub. To troubleshoot issues related to server trust and mapping:

  1. View Tanium Server trust history to see details about who approved or revoked trust, when, and whether the operation succeeded.

  2. Review the PKI log for additional information about events that relate to server trust. See Tanium Core Platform Deployment Reference Guide: PKI log

Troubleshoot certificate and key issues

To troubleshoot errors that relate to the digital certificates and keys that secure communication among Tanium Core Platform servers and Tanium Clients, review the following issues and remediation tasks:

Troubleshoot API token access issues

To troubleshoot issues relating to the Tanium REST API authentication tokens that enable users and service accounts to establish long-lived sessions with the Tanium Server:

  1. View API token details to verify that tokens enable authentication from the correct IP addresses and have not expired.

  2. Tanium Core Platform Deployment Reference Guide: Authentication log

Configure server logging levels

The logging level determines how much detail the Tanium Server and Tanium Module Server record in their logs. Tanium Support might instruct you to change the logging level when troubleshooting issues.

Only users who are assigned the Administrator reserved role can access the Logging page.

For details about logging levels, best practice levels for various use cases, and to change the level for all Tanium Core Platform components, see Tanium Core Platform Deployment Reference Guide: Logging levels.

  1. From the Main menu, go to Administration > Configuration > Logging.
  2. Set the logging level for each server and click Save.

Troubleshoot Interact issues

Troubleshoot question runtimes

If Tanium Clients answer a question slower than expected, the question might use sensors that have long runtimes. The Tanium Console displays runtime indicators to show the average runtimes of sensors that you select for questions. If necessary, you can customize the thresholds that determine which indicator appears for a specific runtime. See Managing sensor runtime thresholds.

Troubleshoot question results issues

If Tanium Clients do not answer questions, review the following issues and remediation tasks:

Troubleshoot action deployment issues

To ensure actions deploy as expected and to troubleshoot deployment issues, see Monitor actions.

Collect Interact logs

To send information to Tanium Support for troubleshooting Tanium Interact, perform the following steps to collect logs and other relevant information. The information is saved as a ZIP file that you can download through your browser.

  1. From the Interact Overview page, click Help .
  2. In the Troubleshooting section, click Download Support Package.
    A tanium-interact-support-<date-time>.zip file downloads to the local download directory.
  3. Attach the ZIP file to your Tanium Support case form or send it to Tanium Support.

Manage the package file repository

By default, the Tanium Server stores the package files that it downloads to Tanium Clients in the <Tanium Server installation directory>/Downloads folder. When you are troubleshooting download issues, Tanium Support might instruct you to monitor usage for this repository and to free space in it if necessary.

Only users who have the Administrator reserved role can see the Package File Repository page.

As additional troubleshooting for download issues on the Tanium Server, review the following logs as described in the Tanium Core Platform Deployment Reference Guide:

View package file repository usage

From the Main menu, go to Administration > Configuration > Package File Repository and review the information.


Clean the package file repository (Windows infrastructure only)

By default, the Tanium Server automatically cleans the repository (deletes unused package files) every Sunday at 2 AM. However, if you see symptoms of low disk space on the server, you can manually clean the repository before then if the server is deployed on a Windows host. For example, when space is low, users might not be able to access the Tanium Console sign-in page or they might experience sign-in failures.

To manually clean the repository:

  1. Sign in to the Tanium Server host as the Administrator user.

  2. Open the Windows Command Prompt to access the CLI.

    For information on using the CLI, see Tanium Core Platform Deployment Reference Guide: Command-line interface.

  3. Navigate to the Tanium Server <installation directory>:

    cd <installation directory>

  4. Run the following command to clear the repository:

    TaniumReceiver clean-downloads

Relocate the package file repository (Windows infrastructure only)

Optionally, you can relocate the package file repository if another drive on the Tanium Server host has better resources for storing package downloads. See Tanium Core Platform Deployment Guide for Windows: Relocate the package file repository.

Troubleshoot role-based access control (RBAC) issues

Troubleshoot permission issues

If you see permission errors when trying to access certain Tanium Console pages or features, perform the following tasks.

If you cannot access the Tanium Console, see Troubleshoot Console access.

The browser that you use to access the Console might display the 403 Forbidden error for permission issues.

  1. Determine which permissions are required to access specific pages and features. See User role requirements.

  2. Verify that the required permissions are assigned to the user account or persona that you used when trying to access those pages and features:

  3. If the effective permissions of the user or persona are insufficient, update the role assignments or role configurations. If the user inherits permissions, you might have to assign that user to a different user group.
  4. Review the following logs for additional information about RBAC issues, as described in the Tanium Core Platform Deployment Reference Guide:

    • RBAC log

      The Console error log might record an entry such as the following to identify permission errors:

      ERROR: 400 Bad Request RBAC Exception (Ref# 1f14e8215610cf72): RBACInsufficientPrivilege

      You can use the Ref# value to find the corresponding entry in the RBAC log, which provides additional details about the error. See Work with the Console error log.

    • Module-provided privileges logs

Troubleshoot service account lockouts

If passwords change for service accounts, lockout errors might occur for those accounts until you update the account configurations:

Troubleshoot TDS issues

Monitor resource usage for sensor results collection

The Tanium Data Service (TDS) collects and stores the results of all sensors that are registered for collection so that users can see those results for offline endpoints when issuing questions. Sensor collection consumes resources such as network bandwidth, processing on endpoints, and disk space on the Tanium Server. Resource consumption increases with the cardinality of sensors. For example, the IP Address sensor produces a unique result string for each queried endpoint, whereas the Operating System (OS) sensor produces the same string for all endpoints that have the same OS. In this case, the high cardinality IP Address sensor requires more bandwidth, CPU usage, and storage. Interact provides charts that enable you to visualize resource usage metrics related to results collection.

For more details and procedures related to sensor results collection, see Manage sensor results collection.

  1. Go to the Interact Overview page and click Info Information.
  2. Review the following charts:
    • Harvest Metrics: These charts display metrics related to the number of database rows that are processed when question results are collected for registered sensors. These charts only display when the Continuous Harvest option is selected. For more information, see Configure advanced settings for sensor collection.
    • Data Service Status: This chart displays metrics related to sensor collection processes when the Continuous Harvest option is deselected. By default, the Tanium Data Service runs the Data Collection process every hour to collect results for registered sensors and runs the Garbage Collection process every 15 minutes to remove expired result strings. The chart uses the following icons. Hover over an icon to display details about a specific process instance.
      • Success: process that completed successfully
      • Refresh: process that is currently running
      • Error: process with errors
      • Future: pending process
    • Data Service Sensor Metrics: Use these charts to determine whether specific sensors are generating result strings that consume too much storage.
    • Data Service Database Metrics: These charts provide indicators on the disk space usage for the Tanium Data Service. The Database Key Size and Database Value Size charts show usage in bytes.
    • Data Service Resource Consumption Metrics: Use these charts to determine the resource usage for the Tanium Data Service.

Resolve TDS resource consumption issues

If you determine that sensor collection consumes too many resources, consider the following solutions:

Troubleshoot Tanium solution issues

Verify and update solution versions

Installing later versions of Tanium solutions might resolve unexpected behavior. The Tanium Console version appears in the Console header (Main menu). To check the current installed versions of other solutions and update them if necessary, see Managing Tanium solutions.

Install the most recent version of every solution.

Troubleshoot installation, update, or uninstallation issues

If errors occur when you install, update, or uninstall Tanium modules, shared services, or content packs, review the following issues and remediation tasks:

If the browser that you use to access the Console displays the 404 Not Found error, solution files might be missing due to an incomplete installation. However, this error might also occur for other reasons, such as browser extension issues.

Troubleshoot plugin issues

A plugin is an extension to a Tanium Core Platform component, module, or shared service. Tanium modules and shared services run scheduled plugins as background processes at intervals. Plugin operations are usually transparent to users. However, Tanium Support might instruct you to review plugin details when troubleshooting unexpected behavior.

Only users who are assigned the Administrator reserved role can access the Configuration pages for viewing plugin information.

To see details about installed plugins and scheduled plugins, from the Main menu go to Administration > Configuration > Plugins. The Plugins page displays separate grids for installed plugins and scheduled plugins.

Perform the following tasks to troubleshoot plugin errors:

  1. Verify that passwords are up-to-date for the service accounts that Tanium solutions use to create plugins. Solutions might create new plugins during version upgrades or other events. See Manage service accounts.
  2. Review the module plugin history logs. See Tanium Core Platform Deployment Reference Guide: Module plugin history logs.
  3. Contact Tanium Support for more help with plugins.

Troubleshoot solution-specific issues

To troubleshoot issues with a specific solution, see the associated user guide:

Troubleshoot Tanium Client issues

The following subsections describe diagnostic and remediation tasks for Tanium Client connectivity, registration, and licensing issues. For other Client troubleshooting tasks, see Tanium Client Management User Guide: Troubleshooting Tanium Clients and Client Management.

The Client Status page displays information about the state of Tanium Client registration and connectivity, and enables you to deploy actions to remediate issues.

You require a role with Client Status read permission is required to see the Client Status page. The Administrator reserved role has this permission.

View the status of Tanium Client registration and communication

  1. From the Main menu, go to Administration > Configuration > Client Status.
  2. (Optional) To display status details only for specific Tanium Clients, edit the default filter settings, such as the registration intervals and connection status.


The following table lists the information that the Client Status page displays for each Tanium Client:

 Table 1: Client Status columns
Column Description
Host Name Endpoint host name.
Network Location (from client) Client IP address returned from a sensor on the client.
Network Location (from server) Client IP address recorded on the Tanium Server or Zone Server during the last registration.
Direction A circle represents the client and arrows represent its connections. For a list of possible connection states, see Table 2.
Valid Key No indicates an issue with the public key that the Tanium Client uses to secure communication with other Tanium Core Platform components. To resolve the issue, reinstall the Tanium Client (see ) or redeploy the key (see ).
Using TLS This column indicates whether clients are (Yes) or are not (No) using Transport Layer Security (TLS) for communication with the Tanium Server or Zone Server.
Send State
  • Normal: The client is sending data to its backward and forward peers.
  • None: The client is not sending data to its forward or backward peers.
  • Forward Only: The client is sending data to its forward peer but not to its backward peer.
  • Backward Only: The client is sending data to its backward peer but not to its forward peer.
Receive State
  • Normal: The client is receiving data from its backward and forward peers.
  • None: The client is not receiving data from its forward or backward peers.
  • Next Only: The client is receiving data from its forward peer but not from its backward peer.
  • Previous Only: The client is receiving data from its backward peer but not from its forward peer.
Status
  • Normal: The client is communicating normally.
  • Slow Link: The client has connections with abnormally slow throughput.
  • Leader: The client is communicating with the Tanium Server or Zone Server because it is a backward leader, a forward leader, a neighborhood leader, or a client with no peer connections (such as a client in an isolated subnet).
  • Blocked: The client is not communicating reliably.
Registration Error (Salesforce deployments only) The client can be in one of the following registration states:
  • None: Registration succeeded
  • Failed Client Challenge: Registration failed because the client did not present the registration secret that is contained in the tanium-init.dat file.
  • Failed Server Challenge: Registration failed because mismatching Tanium root keys prevented Tanium Cloud from establishing trust with the client or other Tanium Core Platform components.
  • Root Key Mismatch: Registration failed due to any other issue related to mismatching Tanium root keys.
  • Exceeded License Seats: Tanium Cloud denied registration for the client because the number of active clients exceeded the limit that the Tanium license specifies. Clients are considered active if they registered within the last two days.

To resolve issues that relate to the Tanium root key or registration secret, re-install the Tanium Client on the affected endpoints. for a new Tanium license if you want to change the maximum number of active clients.

Last Registration Date and time when the Tanium Client last registered with the Tanium Server or Zone Server.
Protocol Version (hidden by default) Tanium Protocol version. For details about the protocol, see TLS communication.
Version Tanium Client version.

The Direction column displays icons that use the following conventions to depict Tanium Client connection states:

  • An up arrow indicates a connection with Tanium Cloud the Tanium Server or Zone Server.
  • Side arrows pointing away from the client indicate outbound connections to peers.
  • Side arrows pointing at the client indicate inbound connections from peers.
  • Side arrows on the right side of clients indicate the state of connections to forward peers.
  • Side arrows on the left side of clients indicate the state of connections to backward peers.
  • Side arrows with dashed lines indicate slow connections.

You can use the Direction column to understand why a Tanium Client is a leader and to identify connection issues. The following table lists the possible connection states:

 Table 2: Tanium Client peer connection states
Attribute Value Description
Leader Backward

Backward leader

The client is a backward leader that terminates one end of a linear chain. It typically has the lowest IP address in its linear chain.
Forward

Forwared leader

The client is a forward leader that terminates one end of a linear chain. It typically has the highest IP address in its linear chain.
Neighborhood

Leader

The client is designated as a neighborhood leader because its linear chain has reached the maximum number of clients.
Isolated

No peering

The client is an isolated leader that connects only to Tanium Cloud the Tanium Server or Zone Server, and has no connections to other clients. The client might be isolated because its IP address falls within the range of an isolated subnet or because it has no peers in its local subnet with which to connect.
Neighbor No side arrows

No peering

This is the same as an isolated leader.
Single side arrow

inbound only or outbound only

The client has a neighborhood list of peers but has not established a peer connection. This state generally results from a misconfiguration, such as when a host-based firewall on the endpoint does not allow inbound connections to the client.
Double side arrows

inbound and outbound connections

The client has a neighborhood list of peers and has connected with peers in the indicated direction.
Client state Normal

inbound and outbound connections

The client is communicating normally.
Blocked

blocked

The client is not communicating reliably. This might result from a network issue or host resource issue, such as an anti-virus program that slows the client.

Export Tanium Client status details

Export information in the Client Status page as a CSV file or, if you have the AdminAdministrator reserved role, as a JSON file.

  1. From the Main menu, go to Administration > Configuration > Client Status.
  2. Select rows in the grid to export information for specific Tanium Clients. If you want to export information for all clients, skip this step.
  3. Click Export Export.
  4. (Optional) Edit the default export File Name.

    The file suffix (.csv or .json) changes automatically based on the Format selection.

  5. Select an Export Data option: information for All clients in the grid or only for the Selected clients.
  6. Select the file Format: JSON (AdminAdministrator reserved role only) or CSV.
  7. Click Export.

    Tanium CloudThe Tanium Server exports the file to the downloads folder on the system that you used to access the Tanium Console.

Copy Tanium Client status details

Copy information from the Client Status page to your clipboard to paste the information into a message, text file, or spreadsheet. Each row in the grid is a comma-separated value string.

  1. From the Main menu, go to Administration > Configuration > Client Status.
  2. Perform one of the following steps:
    • Copy row information: Select one or more rows and click Copy Copy.
    • Copy cell information: Hover over the cell, click Options Options, and click Copy Copy.

Deploy actions to remediate client registration or connectivity issues

You can deploy actions to Tanium Clients to remediate issues that you observe in the Client Status page. For example, if you want certain clients to register with a Tanium Zone Server instead of the Tanium Server, you can deploy the Set Tanium Server Name List package to change the ServerNameList setting on those clients.

  1. From the Main menu, go to Administration > Configuration > Client Status.
  2. Select the Tanium Clients (up to 100) to which you want to deploy actions and click Deploy Action.
  3. Deploy the action.
  4. Review the Client Status grid to verify that the action produced the expected result.

View the info page

If you consult Tanium Support for troubleshooting, Support might instruct you to review settings or counters on the info page.

You require a role with Server Status read permission to view the info page.

  1. Open a browser and go to https://<Tanium Server>/info, where <Tanium Server> is the FQDN or IP address of the Tanium Server.
  2. Sign in as a user who is assigned the Administrator reserved role.





Contact Tanium Support

Tanium Support is your first contact for assistance with preparing for and performing a solution installation or upgrade, as well as verifying and troubleshooting the initial deployment. If you require further assistance from Tanium Support, include version information for Tanium Core Platform components and specific details on dependencies, such as the host system hardware and OS details and database server version. You can also send Tanium Support a collection of logs and other information as a ZIP file: see Collect Interact logs.

To contact Tanium Support for help, sign in to https://support.tanium.com.