Troubleshooting Tanium Clients and Client Management

This section identifies resources that you can use when troubleshooting issues with the Tanium Client and with Client Management.

Basic tips

Client health: Review client health information in the Client Management service to help identify general issues with the Tanium Client on endpoints: see Monitor the client health overview in Tanium Cloud and Access detailed client health and troubleshooting information on an endpoint.
Version: Contact Tanium Support to verify that the Tanium Client version is a recommended version.
Requirements: Ensure your environment meets the host system and network requirements: see Tanium Client and Tanium Cloud requirements.
Access: If you encounter failed access messages when running a Tanium Client installer, examine the permissions for the signed in user.
Installation: For Tanium Client installation issues:
- Examine the Tanium Client installation log on the endpoints.
- Make sure the endpoint has enough available space on the disk or partition where you are installing the client: see Hardware requirements.
Connection and registration: See Troubleshoot issues with connection and registration.
Client settings: See Managing client settings and Index configurations and Tanium Client settings reference.
Actions: For issues that occur when deploying actions, see Review action logs and associated files to troubleshoot actions and packages and Review action history logs to troubleshoot or audit actions.
Sensors: For issues related to sensors, see Review sensor history logs to troubleshoot or audit sensor activity and Review and manage sensor quarantines to troubleshoot sensors.
Tanium Client service: See Verify that the Tanium Client service and process are running on an endpoint.

Review the Tanium Client installation log to troubleshoot installation on Windows

If you encounter issues with your installation on Windows endpoints, examine Install.log in the Tanium Client installation directory to identify actions that failed during the installation. The Tanium Client installer generates this log file to record a chronology of the actions that the installer performed. Each time the installer runs (that is, for each installation), it appends the actions for that execution to the end of the existing log file.

Troubleshoot issues with connection and registration

If the Tanium Client fails to connect or register with Tanium Cloud, does not establish the expected peer connections, or fails to respond to questions, review the Tanium Client logs, and check the following items.

Check the client status

From the Main menu, go to Administration > Configuration > Client Status. Filter the list as necessary to help locate the endpoint.

For more information about the Client Status page, see Verify or remediate Tanium Client peering and leader connections.
If the endpoint does not appear in the current list, select Show systems that have reported in the last, and adjust the time period to determine if the endpoint has previously reported. If the endpoint previously reported, consider whether there were changes near the Last Registration time on the endpoint or the network that might have affected the connectivity of the Tanium Client.
If the endpoint does not appear, or if No appears in the Valid Key column, check the public key (tanium-init.dat) for the client. For more information, see Review or reset the public key to troubleshoot connection issues.

(Salesforce deployments only) The Registration Error column provides additional information if the client failed to register. For more information, see View the status of Tanium Client registration and communication.
If the endpoint is not currently reporting and the client appears to have a valid key, proceed to the next troubleshooting task.

Verify that the Tanium Client service and process are running on an endpoint

Check the status of the Tanium Client service and, if necessary, restart it:

Additionally you can use the following commands to verify that the Tanium Client process is running:

Windows: tasklist | findstr /i "TaniumClient"
Non-Windows: ps -eaf | grep -i TaniumClient

If the Tanium Client service, process, or installation directory does not exist, reinstall the Tanium Client. For more information, see Deploying the Tanium Client using an installer or package file.

Verify port accessibility and security exclusions

Make sure that communication on port 17472 is allowed by any firewalls and other security applications.

Make sure that security exclusions are in place for Tanium Client directories and processes. For more information, see Security exclusions for Tanium Client.

Verify Tanium Cloud connection settings

For Tanium Cloud connection issues, use the following commands to review and verify the server connection settings for the client.

Setting	OS
Tanium Cloud FQDNs	Windows	TaniumClient config get ServerNameList
Tanium Cloud FQDNs	Non-Windows	sudo ./TaniumClient config get ServerNameList
Proxy servers (where used)	Windows	TaniumClient config get ProxyServers
Proxy servers (where used)	Non-Windows	sudo ./TaniumClient config get ProxyServers
Proxy auto configuration (PAC) file (where used)	Windows	TaniumClient config get ProxyAutoConfigAddress

If any settings are incorrect, or for more information about Tanium Cloud connections, see Configuring connections to the Tanium Core Platform.

For peer connection issues, see Configuring Tanium Client peering.

Test DNS resolution

Use the following command to test the DNS resolution for each Tanium Cloud FQDN that is specified for ServerNameList:

nslookup <Tanium Cloud_FQDN>

If the command does not return one or more IP addresses for the Tanium Cloud FQDN, there is likely an issue with DNS resolution. Work with your network administrator to resolve the issue.

Test network connectivity and port accessibility

To verify that the endpoint can communicate with port 17472 on a Tanium Cloud FQDN, use one of the following commands:

Windows PowerShell:Test-NetConnection -ComputerName <Tanium Cloud_FQDN> -Port 17472
Non-Windows:nc -vz <Tanium Cloud_FQDN> 17472

If the connection fails, work with you network administrator to make sure that your Tanium Cloud FQDNs are reachable from your network, and that connections to those FQDNs and communication on port 17472 are allowed by any firewalls and other security applications.

Collect troubleshooting information from endpoints

You can use Client Management to directly connect to an endpoint and collect a bundle of logs and other artifacts, sometimes referred to as an Endpoint Must Gather (EMG).

From the Main menu, click Administration > Shared Services > Client Management.
From the Client Management menu, click Client Health.
In the Direct Connect search box, enter all or part of an IP address or a computer name.

Matching results are displayed after the search completes.
From the search results, click the computer name to connect to the endpoint.
Click the Gather tab. In the Domain section, select the category or Tanium Solution for which you want to gather troubleshooting information.
Click Gather from Endpoint.

The selected logs and artifacts are gathered from the endpoint. The package appears in the Must Gathers section, and the name of the package corresponds with its time stamp.
When Finished appears in the Run State column, select the package and click Download to download a ZIP file that contains the troubleshooting information.

Access individual endpoint logs in Client Management

You can use Client Management to directly connect to an endpoint and view and download individual logs.

From the Main menu, click Administration > Shared Services > Client Management.
From the Client Management menu, click Client Health.
In the Direct Connect search box, enter all or part of an IP address or a computer name.

Matching results are displayed after the search completes.
From the search results, click the computer name to connect to the endpoint.
Click the Logs tab, and select a log to view.
(Optional) To download the log, click Download.

Review Tanium Client logs to troubleshoot connections and other client issues

Review Tanium Client logs to help you troubleshoot client issues. For example, a client might not answer questions or appear in the Tanium Console (Administration > Configuration > Client Status) because that client cannot connect to the Tanium Cloud. In this case, you can review the client logs to determine whether the connection failed due to an invalid Tanium Cloud FQDN, DNS resolution failure, missing Tanium public key file, or firewall rule.

The Tanium Client writes new client logs to the file log0.txt. The default maximum log file size is 10 MB. When log0.txt reaches the maximum size, the client renames it log1.txt and then creates a new log0.txt. When log0.txt again reaches the maximum, the client renames log1.txt as log2.txt, again renames log0.txt as log1.txt, and again creates a new log0.txt. The process of rolling logs whenever log0.txt reaches the maximum size continues until 10 logs exist: log0.txt to log9.txt.

When log0.txt reaches the maximum size again after that, the client compresses log9.txt as a file named log10.zip. When log0.txt again reaches 10 MB, the client renames log10.zip as log11.zip and again compresses log9.txt as a file named log10.zip. The ZIP file rollover process continues until 10 ZIP files exist, log10.zip to log19.zip. When log0.txt reaches 10 MB again after that, the client creates a new log10.zip without renaming log19.zip as a new file, effectively dropping the old log19.zip information upon renaming log18.zip as the new log19.zip.

The logging level is configurable (see LogVerbosityLevel1). The location for log files is also configurable (see LogPath). The default is <Tanium Client installation directory>/Logs.

You can use Client Management to directly connect to an endpoint and retrieve client logs. For more information, see Access individual endpoint logs in Client Management.

Network Configuration errors reported in the log

The error message Network Config Timed Out or Failed to download netconfig at startup commonly appears when a Tanium Client fails to connect or register with Tanium Cloud. To troubleshoot this error message, see Troubleshoot issues with connection and registration.

Cache-related errors reported in the log

Cache-related errors that are reported in a client log are often caused by low disk space on the endpoint. Make sure the endpoint has enough available space on the disk or partition where the client is installed. For disk space requirements, see Hardware requirements.

On a Linux endpoint, you can move the Tanium Client if the partition where it is installed does not have enough free space. For more information, see Move an existing installation of the Tanium Client on Linux.

Review action logs and associated files to troubleshoot actions and packages

When a package does not seem to work after you deploy it through an action, review action logs and the files associated with the action to help troubleshoot. Each time the Tanium Client receives an action message with an instruction set to execute, the client creates an action log file named Action_<ID>.log, where <ID> is the action identifier. The action log contains the CLI output associated with the action command. The Tanium Client stores any files that are required to deploy an action package in Action_ID directories.

Both action logs and Action_<ID> directories are in the <Tanium Client installation directory>/Downloads directory. The Tanium Client removes action logs from its host after a configurable interval (see Action log and package cleanup).

The Tanium Console displays the Action ID in the Action > Action History and Action Status pages (see Tanium Console User Guide: Deploying actions). The Action Status page provides options for accessing action log information from multiple endpoints: see Tanium Console User Guide: View action status.

Action history logs provide a longer history of which actions a managed endpoint has run, but without the CLI output and other details.

Action_<ID> directories

Each Action_<ID> directory contains all the files that are required to deploy an action package. For example, if you deploy a package that has five files, the Tanium Client places each file in the Action_<ID> directory after it finishes downloading. After all five files download, the action status changes from Preparing Files to Running on the Action Status page. Even if a deployed package has no associated package files, the Tanium Client creates an empty Action_<ID> directory for it. The Tanium Client removes Action_<ID> directories from its host after a configurable interval (see Action log and package cleanup).

Access action logs in Client Management

You can use Client Management to directly connect to an endpoint and view and download individual logs.

From the Main menu, click Administration > Shared Services > Client Management.
From the Client Management menu, click Client Health.
In the Direct Connect search box, enter all or part of an IP address or a computer name.

Matching results are displayed after the search completes.
From the search results, click the computer name to connect to the endpoint.
Click the Actions tab, and select a previously run action for which you want to view the log.
(Optional) To download the log, click Download.

Action log contents

Action logs record each phase of an action:

Downloading Files
During this phase, the action log entry indicates the files are downloading:
2016-11-28 14:12:30 +0000|Downloading Files.
2016-11-28 14:12:30 +0000|Files Failed Verification
Although it appears to be an error condition, the message "Files Failed Verification" indicates simply that the client does not have the necessary files in its local cache, so it asks for the necessary files from its peers. This indicates normal behavior.
Running
During this phase, the action log notes that the action is currently running. Following this entry, the log displays anything echoed from the package:
2016-11-28 14:12:37 +0000|Files Verified, running action.
Completed
When the action finishes running, the log records a completion entry under the standard output capture of the action.
2016-11-28 14:12:37 +0000|Command Completed

Completion does not indicate success. For example, an action to execute a command might complete even if the command itself fails. For example, the command line for the package might not match the name of the distributed file or the command might fail to distribute a file. Managed endpoints show that the action completed, even though nothing occurred. Optionally, consider adding a validation query to the package to have the action status indicate success or failure.

Action log and package cleanup

The Tanium Client checks hourly, or immediately upon resetting (every two to six hours), whether any Action_<ID>.log files are over seven days old and deletes them if they are. The Tanium Client also checks hourly, or immediately upon resetting, whether any corresponding Action_<ID> directories have expired, and deletes them if they have. This process ensures that the endpoint does not consume more disk space than necessary for Tanium actions. Contact Tanium Support if you want to preserve action logs or action directories for a longer time.

Review action history logs to troubleshoot or audit actions

When you troubleshoot or audit actions on managed endpoints, review the action history logs to see which actions ran, their start and run times, and associated commands. Although the Action logs record more details, the Tanium Client preserves action history logs for a longer period (their individual log files are smaller) and therefore they provide a longer chronology of actions. The Tanium Client archives the first 10 MB of action history logs as plain-text files. After reaching the 10 MB threshold, the client archives the oldest logs as ZIP files before adding new logs as plain-text files. The log rollover process is as follows:

Plain text logs files

The Tanium Client creates a new action-history0.txt file whenever an action runs. When that file reaches 1 MB in size, the client renames action-history0.txt as action-history1.txt and creates a new action-history0.txt. When action-history0.txt again reaches 1 MB, the client renames action-history1.txt as action-history2.txt, again renames action-history0.txt as action-history1.txt, and again creates a new action-history0.txt. The process of rolling logs whenever action-history0.txt reaches 1 MB continues until 10 logs exist: action-history0.txt to action-history9.txt.

ZIP log files

After recording 10 MB of plain-text action history logs, the Tanium Client compresses action-history9.txt as a file named action-history10.zip. When action-history0.txt again reaches 1 MB, the client renames action-history10.zip as action-history11.zip and again compresses action-history9.txt as a file named action-history10.zip. The ZIP file rollover process continues until 10 ZIP files exist, action-history10.zip to action-history19.zip. When action-history10.zip reaches 1 MB again after that, the client creates a new action-history10.zip without renaming action-history19.zip as a new file, effectively dropping the old action-history19.zip information upon renaming action-history18.zip as the new action-history19.zip.

The Tanium Client stores action history logs in the <Tanium Client installation directory>/Logs directory.

You can use Client Management to directly connect to an endpoint and retrieve action history logs. For more information, see Access individual endpoint logs in Client Management.

Review sensor history logs to troubleshoot or audit sensor activity

When you troubleshoot or audit sensor activity on managed endpoints, review the sensor history logs to see the following information about each sensor that ran:

Sensor identity, by name and hash value
Start and run times
Size in bytes
Parameter values (the logs identify parameterized sensors as temp sensors)
Number of answer strings and associated hash value

The Tanium Client archives the first 10 MB of sensor history logs as plain-text files. After reaching the 10 MB threshold, the client archives the oldest logs as ZIP files before adding new logs as plain-text files. The log rollover process is as follows:

Plain text logs files

The Tanium Client creates a new sensor-history0.txt file each time a sensor runs. When that file reaches 1 MB in size, the client renames sensor-history0.txt as sensor-history1.txt, and creates a new sensor-history0.txt. When sensor-history0.txt again reaches 1 MB, the client renames sensor-history1.txt as sensor-history2.txt, again renames sensor-history0.txt as sensor-history1.txt, and again creates a new sensor-history0.txt. The process to roll the logs whenever sensor-history0.txt reaches 1 MB continues until 10 logs exist: sensor-history0.txt to sensor-history9.txt.

ZIP log files

After recording 10 MB of plain-text sensor history logs, the Tanium Client compresses sensor-history9.txt as a file named sensor-history10.zip. When sensor-history0.txt again reaches 1 MB, the client renames sensor-history10.zip as sensor-history11.zip and again compresses sensor-history9.txt as a file named sensor-history10.zip. The ZIP file rollover process continues until 10 ZIP files exist, sensor-history10.zip to sensor-history19.zip. When sensor-history10.zip reaches 1 MB again after that, the client creates a new sensor-history10.zip without renaming sensor-history19.zip as a new file, effectively dropping the old sensor-history19.zip information upon renaming sensor-history18.zip as the new sensor-history19.zip.

The Tanium Client stores sensor history logs in the <Tanium Client installation directory>/Logs directory.

You can use Client Management to directly connect to an endpoint and retrieve sensor history logs. For more information, see Access individual endpoint logs in Client Management.

Review or reset the public key to troubleshoot connection issues

You can review or reset the public key to help resolve connection issues that are related to an invalid key.

(Salesforce deployments only) The Registration Error column on the Client Status page indicates specific issues with keys. For more information, see View the status of Tanium Client registration and communication.

Access the operating system CLI on the endpoint and change directory (cd) to the Tanium Client installation directory.
Enter the following command:
- Windows: TaniumClient pki show
- Non-Windows: ./TaniumClient pki show
The output displays information about the current public key. Make sure that the command returns licenses for the appropriate Tanium Cloud instances, the status for each Tanium Cloud instance is trusted, and the fingerprint for each license matches the fingerprint in Tanium Cloud. For more information, see Tanium Console User Guide: Managing Tanium keys.
(Optional) Reset the key with a new tanium-init.dat file.
1. From the Main menu in the Tanium console, go to Administration > Shared Services > Client Management.
2. From the Client Management Overview page, download the installation package for the OS of the endpoint.
3. Extract the tanium‑init.dat file from the downloaded bundle, and copy it into the Tanium Client installation directory.
4. From the CLI on the endpoint, enter the following command:
  - Windows: TaniumClient pki reset tanium-init.dat
  - Non-Windows: ./TaniumClient pki reset tanium-init.dat
Be careful not to allow the tanium-init.dat file to be distributed or stored outside of your organization, such as in a publicly accessible source code repository or any other location accessible from the public internet. Limit the distribution to specific use in the deployment of Tanium Clients.
Though this file does not contain private keys and cannot be used to provide control over a Tanium environment, a user with malicious intent could use it to connect an unapproved client and use this unauthorized access to learn how your organization is using Tanium.

Review and manage sensor quarantines to troubleshoot sensors

Enforcing sensor quarantines prevents sensors from running on an endpoint for the current question or action if those sensors exceeded the runtime timeout during a previous question or action. Quarantines are useful for limiting the impact on endpoint resources, such as CPU utilization, when questions and actions use excessively long-running sensors. The non-configurable timeout is set to one minute.

By default, quarantines are not enforced: after a sensor exceeds the timeout and stops running, the sensor has quarantined status but still runs for future questions or actions until it completes or times out. In this case, the Tanium Client uses the quarantined status just to record that the sensor timed out.

Regardless of whether you enable enforcement, the Tanium Client stops any sensor at the moment it exceeds the timeout. Each client quarantines sensors and enforces the quarantines independently. Consequently, a sensor might be quarantined on some endpoints and not on others.

When a Tanium Client quarantines a sensor, the Tanium Console displays the following message in the Question Results grid: TSE-Error: Sensor evaluation timed out. When you issue a question that uses a sensor that is already quarantined and enforcement is enabled, the Question Results grid displays TSE-Error: The sensor is quarantined. The Tanium Client adds entries to the client logs and sensor history logs when it quarantines a sensor or prevents an already quarantined sensor from running.

If temporary sensors exceed the one-minute timeout, the Tanium Client quarantines the original sensor as well as all current and future temporary sensors that are based on the original sensor.

When enforcement is enabled, quarantined sensors do not run when you use them for targeting endpoints, even if the sensors are members of computer groups. However, quarantined sensors might skew the targeting of a question that has a vague from clause, such as from all machines with Is Windows not equals true. In this case, Windows endpoints on which the Is Windows sensor is quarantined would match the condition not equals true because their response would be TSE-Error: The sensor is quarantined rather than true. To avoid such outcomes, make the target clause as specific as possible and do not use negative matching conditions such as not equals true.

View quarantined sensors

If the Tanium Client does not answer a question, you can determine whether the associated sensors are quarantined. To see a list of all the quarantined sensors on all endpoints, see Tanium Console User Guide: Manage sensor quarantines. To list all the quarantined sensors on a specific endpoint, perform the following steps:

Access the operating system CLI on the endpoint and change directory (cd) to the Tanium Client installation directory.
Enter the following command.
- Windows: TaniumClient quarantine list
- Non-Windows: ./TaniumClient quarantine list
The output lists the quarantined sensors by name and associated hash value.

Remove all sensors from quarantine

In some cases, enabling the Tanium Client to answer questions that use quarantined sensors might be more important than limiting the impact that long sensor run times have on the resources of an endpoint. Note that even after you remove the sensors from quarantine, if they exceed the timeout in a future question, the Tanium Client will then stop the sensors and quarantine them again without answering the question. To remove sensors from quarantine through the Tanium Console, see Tanium Console User Guide: Manage sensor quarantines. To remove sensors from quarantine through the operating system CLI on the endpoint, perform the following steps:

Access the operating system CLI on the endpoint and change directory (cd) to the Tanium Client installation directory.
Enter the following command:
- Windows: TaniumClient quarantine clear
- Non-Windows: ./TaniumClient quarantine clear
The output displays the number of sensors removed from quarantine.

Remove a single sensor from quarantine

To remove a sensor from quarantine through the Tanium Console, see Tanium Console User Guide: Manage sensor quarantines. To remove a sensor from quarantine through the operating system CLI on the endpoint, perform the following steps:

Access the operating system CLI on the endpoint and change directory (cd) to the Tanium Client installation directory.
Enter the following command to see the hash values associated with quarantined sensors.
- Windows: TaniumClient quarantine list
- Non-Windows: ./TaniumClient quarantine list
Enter the following command, where <sensor_hash> is the hash associated with the sensor that you want to unquarantine:
- Windows: TaniumClient quarantine remove <sensor_hash>
- Non-Windows: ./TaniumClient quarantine remove <sensor_hash>

If you modify a sensor, Tanium Clients that receive its new definition automatically remove that sensor from quarantine.

Add a sensor to quarantine

You can manually quarantine a sensor on an endpoint if you anticipate that running the sensor will negatively affect the endpoint.

Quarantining a sensor does not automatically enable quarantine enforcement.

In the URL field of the browser that you use to access the Tanium Console, enter https://<Tanium Cloud Client Edge URL>/hash/<sensor>. The <sensor> must match the sensor name that the Tanium Console displays with respect to capitalization and spaces.

The browser displays the hash value associated with the sensor.
Access the operating system CLI on the endpoint and change directory (cd) to the Tanium Client installation directory.
Enter the following command.
- Windows: TaniumClient quarantine add <sensor_hash>
- Non-Windows: ./TaniumClient quarantine add <sensor_hash>

Enable or disable enforcement of quarantined sensors

After you enable quarantine enforcement, Tanium Clients do not answer questions that use quarantined sensors and those sensors do not run for actions. After you disable enforcement, clients still quarantine sensors and log quarantine events, but do not prevent those sensors from running.

Add or edit the EnableSensorQuarantine setting on the Tanium Clients for which you want to enable or disable quarantine enforcement.

Contact Tanium Support

Tanium Support is your first contact for help when troubleshooting the initial deployment and for optimizing the speed and scale of your deployment as the number of managed endpoints grows. As necessary, Tanium Support can help adjust Tanium Client-related settings, including:

Tanium Client registration frequency
Connections between Tanium Clients and Tanium Cloud
Client-to-client connections
Bandwidth
File caching

If you require further assistance from Tanium Support, include Tanium Client and, if applicable, Tanium Client Management version information. Also include specific details on dependencies, such as the host system hardware and OS details. Finally, indicate if your installation uses a non-default installation directory for the Tanium Client.

To contact Tanium Support for help, sign in to https://support.tanium.com.