Troubleshooting

This chapter covers procedures, settings, and logs that you can use to troubleshoot issues relating to the installation or upgrade of Tanium Core Platform servers.

Basic troubleshooting tips

If the browser that you use to access the Tanium Console displays an HTTP response status code in the 500 to 599 range, this indicates a server error. The responsible server might or might not be a Tanium Core Platform server.

Back up Tanium Core Platform servers and databases

Back up the Tanium Core Platform servers and Tanium databases so that you can restore your deployment to a known functional state in case of issues. For example, a system failure might make the host system of the Tanium Server unrecoverable.

Best practices for backups

The following are standard guidelines for backups. Your backup strategy might differ based on the specific needs of your organization.

  • Define a disaster recovery (DR) plan early during the initial deployment phase. Define specific restore time objectives (RTO) and restore point objectives (RPO), which affect how frequently you back up your deployment.
    • RPO: The expected point of recovery. For example, an RPO of 1 day means recovery point within 24 hours of the time of failure.
    • RTO: The amount of downtime that is acceptable to recover the system.

    Tanium data, such as question result strings, change frequently. Therefore, back up your deployment daily to ensure that you do not lose the latest state that critical operations require.

  • Back up your deployment before any major changes, including upgrades to Tanium Core Platform servers or migrations to new servers.
  • If you rotate the Tanium root keys, back up your deployment immediately after revoking the old keys. For details about managing the root keys, see Tanium Console User Guide: Managing Tanium keys.

    If you configure the Tanium Server to integrate with an HSM, you cannot use a backup of the pki.db file containing the root keys to install a cryptographically identical duplicate of the server.

  • Archive, secure, and manage copies of the SSL/TLS certificate and key files according to the security policies of your organization as you would any other system-level security and credential files. See Tanium Core Platform Deployment Reference Guide: Securing Tanium Console, API, and Module Server access

Back up the servers and database

  1. Copy the installation directories for Tanium Core Platform servers and their associated databases to a secure location. Ensure that you can access the backup location in the event that a disaster makes the server hosts inaccessible.

    The following are the default installation directories for Tanium Core Platform servers:

     Table 1: Default installation directories for Tanium Core Platform servers
    Target ServerInstallation directory
    1Tanium Server\Program Files\Tanium\Tanium Server
    Tanium Module Server\Program Files\Tanium\Tanium Module Server
    \Program Files\Tanium\Tanium Module Postgres
    2Tanium Zone Server\Program Files (x86)\Tanium\Tanium ZoneServer
    3Tanium Zone Server Hub\Program Files (x86)\Tanium\Tanium ZoneServer

    1 Back up this directory on each Tanium Server in an active-active deployment. You might also have to copy the Tanium Server Downloads directory if it was moved out of the installation directory using the instructions in the KB article Relocate Downloads Directory.

    2Backing up the Zone Server and hub is not essential because their data comes from the Tanium Server. After restoring from backups, you would have to re-establish trust between a Tanium Server and Zone Server anyway. To recreate the Zone Server and hub, you can also reinstall them.

    3 The Zone Server Hub can be on the same host as the Tanium Server or on a dedicated host.

  2. Back up the Windows registry keys for the Tanium Core Platform.
    1. Open the Windows Registry Editor.
    2. Navigate to the Tanium registry key, right click the Tanium registry key, and select Export.Back up Tanium registry
    3. Set the Save in field to the backup location, enter a backup File name, and click Save.
  3. Back up the tanium and tanium_archive databases.

    To automate Microsoft SQL (MSSQL) database backups, see the Microsoft SQL documentation.

    To manually back up an MSSQL database through Microsoft SQL Server Management Studio:

    1. Navigate to the database in the Object Explorer.
    2. Right-click the database name, select Tasks > Back Up, and follow the prompts.

    Contact Tanium Support for the steps to back up Tanium databases on a PostgreSQL server.

Common issues and solutions

The following table lists issues that you might encounter when installing or upgrading Tanium Core Platform servers, as well as recommended troubleshooting steps:

 Table 2: Common deployment issues and solutions
Issue Troubleshooting steps
Failed access messages If you encounter failed access messages when running an installer, examine the permissions for your user account. See Administrator account permissions.
Service account lockouts If passwords change for the service accounts that Tanium Core Platform servers use, lockout errors might occur until you update the passwords in the Windows Services program. See Tanium Console User Guide: Windows service accounts.
Failed connections See Troubleshoot connectivity among core platform servers.
Tanium Console is unavailable

See Tanium Console User Guide: Troubleshoot Console access.
Disk space depletion

To monitor and free disk space in the directories where the Tanium Server stores files that it deploys to Tanium Clients, see:
Installation verification fails If you encounter verification failures while performing the steps under Verifying the deployment, see Troubleshoot server installation and upgrade issues.
Installer files or other Tanium executables do not behave as expected If a Tanium executable file that you manually downloaded does not behave as expected, you can verify the file hash as an integrity check. See Verify file integrity.

Troubleshoot server installation and upgrade issues

Perform the following tasks to troubleshoot installation and upgrade issues for Tanium Core Platform servers. The tasks are listed in the order that issues might occur during the installation or upgrade and in the order that you perform subsequent diagnostics and remediation.

  1. If a server installer file that you manually downloaded does not behave as expected, verify its file hash to ensure that it has not been compromised. See Verify file integrity.

  2. If you encounter failed access messages when running a server installer, examine your Administrator account permissions. To change accounts or update their passwords, see Tanium Console User Guide: Windows service accounts. After updating account permissions or passwords, re-run the server installer.
  3. If other issues occur when you run a server installer, review the following logs:

    • All servers:

      • Installation logs: Record events that server installers perform during installations and upgrades.

      • Database upgrade log: Records actions that the Tanium Server installer performs on Tanium database schemas when you upgrade the Tanium Core Platform.
      • Server logs: Record events that the other log types do not capture.
    • Module Server: The PostgreSQL installation log records events that relate to the local PostgreSQL instance that the Module Server installer automatically creates.
  4. After you install or upgrade all the Tanium Core Platform servers, verify that the operation succeeded:

    If a verification step fails, perform the appropriate troubleshooting task:

  5. Access the CLI of a platform server to review and, if necessary, correct server settings. See Review and update server settings.
  6. Troubleshoot connectivity among core platform servers.
  7. If the preceding steps do not resolve the issue, generate logs at logging level 41 and then contact Tanium Support for help analyzing the logs. See Generate and review logs and Contact Tanium Support.

Verify file integrity

If a Tanium executable file that you manually downloaded does not behave as expected, you can use the Tanium™ KeyUtility program to verify the file hash as an integrity check. For example, if you use a token to download the Tanium Server installer (SetupServer.exe), you can calculate the SHA‑256 digest of the installer and compare it to the digest that the tokens site provides for that file. KeyUtility and its associated files reside in the top-level installation directory of the Tanium Server. You can run the program from the installation directory or copy the files to a working directory on another system and run the program from that working directory.

  1. (Optional) To run KeyUtility from a working directory instead of the installation directory:
    1. Copy the following files from the Tanium Server installation directory or from the location where you unzipped the KeyUtility-<release>.zip file that Tanium Support provided:
      • KeyUtility.exe
      • libeay32.dll
      • ssleay32.dll
    2. Paste the files into the working directory.
  2. Open the Windows Command Prompt (cmd.exe) and go to the directory where the KeyUtility and key files reside.

  3. Run the following command, where <algorithm> is the hash algorithm. The supported algorithms are SHA-1, SHA-256, and SHA-512.

    KeyUtility.exe calchash <algorithm> <file>

    The following is an example of the command:

    KeyUtility.exe calchash SHA-256 SetupServer.exe

    The following digest is an example of the output:

    214d00fc4c3d5d52fe0f7a58df566385bd6ad39f5457829e73ead26778e79caf

Manage Windows services for core platform servers

The Tanium Server, Tanium Module Server, Tanium Zone Server, and Tanium Zone Server Hub all have Windows services that must be started for the servers to function and for Tanium Clients to register. To check the status of the services and to stop, start, or restart them:

  1. Sign in to the server for which you want to manage the Windows service.

    Typically, the Zone Server Hub service runs on the Tanium Server, but it can also be on a dedicated machine. Wherever the hub is installed, its service name (Tanium ZoneServer) is the same as for the Zone Server.

  2. Open the Windows Services program, scroll to the service Name, and check its Status.
  3. Click the Stop, Start, or Restart link to perform the corresponding operation.

Review and update server settings

Many settings for Tanium Core Platform servers are added to the Windows Registry when you install the servers. If you encounter issues when installing a server, use the CLI to review the server settings for incorrect values and update them as necessary:

  1. Access the server CLI. See Tanium Core Platform Deployment Reference Guide: Windows: CLI.

  2. List the server settings:

    • Tanium Server: TaniumReceiver config list

    • Module Server: TaniumModuleServer config list

    • Zone Server: TaniumZoneServer config list

    • Zone Server Hub: TaniumZoneServer config list

    If you configured a proxy server, its settings are listed with the Tanium Server and Module Server settings.

  3. Verify that the values are correct based on the setting descriptions in the Tanium Core Platform Deployment Reference Guide:
  4. Update the settings as necessary. The specific command varies by server and setting. For examples, see Tanium Core Platform Deployment Reference Guide: CLI examples.

Troubleshoot connectivity among core platform servers

  1. Use any appropriate utility to test connectivity from the Zone Server Hub to the Zone Server and from the Module Server to the Tanium Server. The following example shows how to use Portqry at the CLI of the Zone Server Hub host to verify whether the Zone Server is listening on a specified port.

    c:\> portqry -n zs1.example.com -p tcp -e 17472
    Querying target system called:
    zs1.example.com
    Attempting to resolve name to IP address...
    Name resolved to 10.10.10.15
    querying...
    TCP port 17472 (unknown service): LISTENING

    If you can reach the server and get an answer (LISTENING), then basic connectivity is not the issue. If you cannot reach the server, you might need to work with your network and security administrators to resolve the issue.

  2. Verify that the server FQDNs resolve to an IP address through Domain Name System (DNS). The preceding step uses Portqry as an example to show DNS resolution. You can also use nslookup at the CLI:

    c:\> nslookup ts1.example.com
    Server: Unknown
    Address: 10.10.10.10
    Name: ts1.example.com
    Address: 10.10.10.15

    If DNS resolution fails, work with your network administrator to resolve it. If DNS resolution is not possible, you can reconfigure the connection settings using IP addresses instead of FQDNs.

  3. Verify that your deployment meets the Internet access, network connectivity, and firewalls requirements.
  4. Review the PKI TLS logs for messages that relate to Transport Layer Security (TLS) connections among Tanium Core Platform components. See Tanium Core Platform Deployment Reference Guide: PKI TLS logs.

Generate and review logs

The default logging level is 1 on Tanium Core Platform servers and Tanium Clients. However, because the servers and clients record certain events only at higher logging levels, you can raise the levels during troubleshooting to generate more detailed logs. If you cannot resolve deployment issues through other troubleshooting steps, perform the following steps to generate and review logs.

For details about logging levels, see Tanium Core Platform Deployment Reference Guide: Logging levels.

If you want to review only specific information in predefined Tanium logs, you can configure a Tanium Core Platform server or Tanium Client to filter the logs based on a regular expression and to copy the matching content to a custom log. Custom logs are especially useful if you set a high logging level for the predefined logs such that they roll over too quickly and record too much information for you to easily find specific issues. See Tanium Core Platform Deployment Reference Guide: Create a custom log.

Contact Tanium Support for help in analyzing the logs.

The following steps describe how to generate logs at logging level 41, which is the best practice when performing temporary troubleshooting for specific issue.

  1. Set the logging level to 41 on:

  2. Sign in to the Tanium Console and repeat the activity that produced the issue that you are troubleshooting.

    For example, import a solution if the previous attempt to import it failed.

  3. Review the logs that pertain to the deployment issue that you are troubleshooting:

    Issue Logs
    Trust among Tanium Servers, Zone Servers, and Zone Server Hubs

    PKI log

    TLS communication among Tanium Core Platform servers and Tanium Clients PKI TLS log
    Zone Server or Tanium Client registration
    Package downloads
    Solution imports
    Other deployment issues Server logs

    4. After you finish troubleshooting, set the logging level to 11 or lower.

Uninstall Tanium

Uninstall the Tanium Core Platform

If you no longer want to use the Tanium Core Platform, or you want to clean up completely before reinstalling:

  1. Uninstall the Tanium Core Platform servers: Tanium Server, Tanium Module Server, Tanium Zone Server, and Tanium Zone Server Hub. The order in which you uninstall the servers does not matter.
  2. Remove the Tanium databases (tanium and tanium_archive) from the Tanium database server. For the specific steps, see your database server documentation.

Uninstall a Tanium Core Platform server

  1. Open the Windows Control Panel and use the Uninstall a program feature to uninstall a Tanium Core Platform server.

    The Windows program invokes the Tanium uninstaller, which stops and removes the Tanium service associated with the server and deletes Windows Registry entries (except the top entry for Tanium). Completely wiping the installation requires the following manual tasks that the installer does not perform.

  2. In the Windows Registry, go to HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\ and delete Tanium.
  3. In Windows Explorer, go to the Tanium installation location and delete the Tanium directory.
  4. Empty the Windows Recycle Bin.

Remove Tanium databases

Sign in to the database server as a database administrator and delete the tanium and tanium.archive databases.



Contact Tanium Support

Tanium Support is your first contact for assistance with preparing for and performing an installation or upgrade, as well as verifying and troubleshooting the initial deployment. If you require further assistance from Tanium Support, please be sure to include version information for Tanium Core Platform components and specific details on dependencies, such as the host system hardware and OS details and database server version.

To contact Tanium Support for help, sign in to https://support.tanium.com.