Troubleshooting

This chapter covers procedures, settings, and logs that you can use to troubleshoot issues relating to the installation or upgrade of Tanium Core Platform servers.

Basic troubleshooting tips

• Regularly run and review Tanium™ Health Check reports, such as the Tanium Platform Analyzer (TPAN) report, regardless of whether you currently have issues to troubleshoot. If necessary, you can share these reports with Tanium Support for troubleshooting assistance. See Tanium Core Platform Deployment Reference Guide: Troubleshooting workflow.
• Ensure that all Tanium Core Platform servers are the same version. For example, make sure all have build number 7.5.5.1162. To update the versions, see Upgrading Tanium Core Platform servers.
• Ensure that your environment meets the host system and network requirements. See Requirements.
• Verify the deployment after you finish installing all the servers. See Verifying the deployment.
• Review any error messages in the:
  • Tanium Console. See Tanium Console User Guide: Work with the Tanium Console error log.
  • Installation log files. See Tanium Core Platform Deployment Reference Guide: Windows logs.

If the browser that you use to access the Tanium Console displays an HTTP response status code in the 500 to 599 range, this indicates a server error. The responsible server might or might not be a Tanium Core Platform server.

Back up Tanium Core Platform servers and databases

Back up the Tanium Core Platform servers and Tanium databases so that you can restore your deployment to a known functional state in case of issues. For example, a system failure might make the host system of the Tanium Server unrecoverable.

Best practices for backups

The following are standard guidelines for backups. Your backup strategy might differ based on the specific needs of your organization.

• Define a disaster recovery (DR) plan early during the initial deployment phase. Define specific restore time objectives (RTO) and restore point objectives (RPO), which affect how frequently you back up your deployment.
  • RPO: The expected point of recovery. For example, an RPO of 1 day means recovery point within 24 hours of the time of failure.
  • RTO: The amount of downtime that is acceptable to recover the system.

  Tanium data, such as question result strings, change frequently. Therefore, back up your deployment daily to ensure that you do not lose the latest state that critical operations require.

• Back up your deployment before any major changes, including upgrades to Tanium Core Platform servers or migrations to new servers.
• If you rotate the Tanium root keys, back up your deployment immediately after revoking the old keys. For details about managing the root keys, see Tanium Console User Guide: Managing Tanium keys.

  If you configure the Tanium Server to integrate with an HSM, you cannot use a backup of the pki.db file containing the root keys to install a cryptographically identical duplicate of the server.

• Archive, secure, and manage copies of the SSL/TLS certificate and key files according to the security policies of your organization as you would any other system-level security and credential files. See Tanium Core Platform Deployment Reference Guide: Securing Tanium Console, API, and Module Server access

Back up the servers and database

1. Copy the installation directories for Tanium Core Platform servers and their associated databases to a secure location. Ensure that you can access the backup location in the event that a disaster makes the server hosts inaccessible.

   The following are the default installation directories for Tanium Core Platform servers:

    Table 1: Default installation directories for Tanium Core Platform servers

   Target Server	Installation directory
   1Tanium Server	\Program Files\Tanium\Tanium Server
   Tanium Module Server	\Program Files\Tanium\Tanium Module Server \Program Files\Tanium\Tanium Module Postgres
   2Tanium Zone Server	\Program Files (x86)\Tanium\Tanium ZoneServer
   3Tanium Zone Server Hub	\Program Files (x86)\Tanium\Tanium ZoneServer
   1 Back up this directory on each Tanium Server in an active-active deployment. You might also have to copy the Tanium Server Downloads directory if it was moved out of the installation directory using the instructions in the KB article Relocate Downloads Directory. 2Backing up the Zone Server and hub is not essential because their data comes from the Tanium Server. After restoring from backups, you would have to re-establish trust between a Tanium Server and Zone Server anyway. To recreate the Zone Server and hub, you can also reinstall them. 3 The Zone Server Hub can be on the same host as the Tanium Server or on a dedicated host.

2. Back up the Windows registry keys for the Tanium Core Platform:
   1. Open the Windows Registry Editor.
   2. Navigate to the Tanium registry key, right click the Tanium registry key, and select Export.
   3. Set the Save in field to the backup location, enter a backup File name, and click Save.
3. Back up the tanium and tanium_archive databases.

   To automate Microsoft SQL (MSSQL) database backups, see the Microsoft SQL documentation.

   To manually back up an MSSQL database through Microsoft SQL Server Management Studio:

   1. Navigate to the database in the Object Explorer.
   2. Right-click the database name, select Tasks > Back Up, and follow the prompts.
      
      

   Contact Tanium Support for the steps to back up Tanium databases on a PostgreSQL server.

Troubleshoot server installation and upgrade issues

Perform the following tasks to troubleshoot installation and upgrade issues for Tanium Core Platform servers. The tasks are listed in the order that issues might occur during the installation or upgrade and in the order that you perform subsequent diagnostics and remediation.

1. If a server installer file that you manually downloaded does not behave as expected, verify its file hash to ensure that it has not been compromised. See Verify file integrity.

2. If you encounter failed access messages when running a server installer, examine your Administrator account permissions. To change accounts or update their passwords, see Tanium Console User Guide: Windows service accounts. After updating account permissions or passwords, re-run the server installer.
3. If other issues occur when you run a server installer, review the following logs:

• All servers:

  • Installation logs: Record events that server installers perform during installations and upgrades.

  • Database upgrade log: Records actions that the Tanium Server installer performs on Tanium database schemas when you upgrade the Tanium Core Platform.
  • Server logs: Record events that the other log types do not capture.
• Module Server: The PostgreSQL installation log records events that relate to the local PostgreSQL instance that the Module Server installer automatically creates.
4. After you install or upgrade all the Tanium Core Platform servers, verify that the operation succeeded:
   • Installation: Verifying the deployment
   • Upgrade: Verify the servers upgrade

   If a verification step fails, perform the appropriate troubleshooting task:

   • If you cannot access the Console, see Tanium Console User Guide: Troubleshoot Console access.

   • If Tanium Servers do not connect with a remote Module Server, or Tanium Clients do not register with Tanium Servers or Zone Servers, check the status of the corresponding Windows services and start any services that are not started. See Manage Windows services for core platform servers.

   • If Tanium Clients do not register even though Tanium Server and Zone Server services are running, see Tanium Client Management User Guide: Troubleshoot issues with connection and registration.

5. Access the CLI of a platform server to review and, if necessary, correct server settings. See Review and update server settings.
6. Troubleshoot connectivity among core platform servers.
7. If the preceding steps do not resolve the issue, generate logs at logging level 41 and then contact Tanium Support for help analyzing the logs. See Generate and review logs and Contact Tanium Support.

Manage Windows services for core platform servers

The Tanium Server, Tanium Module Server, Tanium Zone Server, and Tanium Zone Server Hub all have Windows services that must be started for the servers to function and for Tanium Clients to register. To check the status of the services and to stop, start, or restart them:

1. Sign in to the server for which you want to manage the Windows service.

   Typically, the Zone Server Hub service runs on the Tanium Server, but it can also be on a dedicated machine. Wherever the hub is installed, its service name (Tanium ZoneServer) is the same as for the Zone Server.

2. Open the Windows Services program, scroll to the service Name, and check its Status.
3. Click the Stop, Start, or Restart link to perform the corresponding operation.

Troubleshoot connectivity among core platform servers

1. Use any appropriate utility to test connectivity from the Zone Server Hub to the Zone Server and from the Module Server to the Tanium Server. The following example shows how to use Portqry at the CLI of the Zone Server Hub host to verify whether the Zone Server is listening on a specified port.

   c:\> portqry -n zs1.example.com -p tcp -e 17472
   Querying target system called:
   zs1.example.com
   Attempting to resolve name to IP address...
   Name resolved to 10.10.10.15
   querying...
   TCP port 17472 (unknown service): LISTENING

   If you can reach the server and get an answer (LISTENING), then basic connectivity is not the issue. If you cannot reach the server, you might need to work with your network and security administrators to resolve the issue.

2. Verify that the server FQDNs resolve to an IP address through Domain Name System (DNS). The preceding step uses Portqry as an example to show DNS resolution. You can also use nslookup at the CLI:

   c:\> nslookup ts1.example.com
   Server: Unknown
   Address: 10.10.10.10
   Name: ts1.example.com
   Address: 10.10.10.15

   If DNS resolution fails, work with your network administrator to resolve it. If DNS resolution is not possible, you can reconfigure the connection settings using IP addresses instead of FQDNs.

3. Verify that your deployment meets the Internet access, network connectivity, and firewalls requirements.
4. Review the PKI TLS logs for messages that relate to Transport Layer Security (TLS) connections among Tanium Core Platform components. See Tanium Core Platform Deployment Reference Guide: PKI TLS logs.

Uninstall Tanium