Troubleshooting Patch

If Patch is not performing as expected, you might need to do some troubleshooting or change settings. For information about specific error messages, see Reference: Common errors.

To prevent issues that might occur due to endpoint health issues, ensure that endpoints have:
  • At least 5 GB of free space
  • A healthy Windows Update Agent
  • An up-time of less than 90 days

Collect a troubleshooting package

For your own review or to assist support, you can compile Patch logs and files that are relevant for troubleshooting.

  1. Get the Patch log.
    1. On the Patch Overview page, click Help .
    2. Click Collect Troubleshooting Package.

    The log zip file might take a few moments to download. The files have a timestamp with a Patch-YYYY-MM-DDTHH-MM-SS.mmmZ  format.

  2. (Optional) On the endpoint, copy the <Tanium Client>\Patch\scans folder, excluding the CAB file.

Configure endpoint logging

Distribute the Patch - Set Patch Process Options package to your endpoints to change the default logging type and log rotation settings.

  1. Verify that the Patch - Is Process Running sensor returns Yes for your endpoints.
  2. Ensure that the expected endpoints are in the action group.
  3. Verify that the expected endpoints are targeted by a scan configuration.
  4. Check for scan errors on the endpoints targeted by a scan configuration.

Patches are not listed in the Patches view

If you are having difficulty getting patches to appear:

  1. Verify that the Patch - Is Process Running sensor returns Yes for your endpoints.
  2. Ensure that the expected endpoints are in the action group.
  3. Verify that the expected endpoints are targeted by a scan configuration.
  4. Check for scan errors on the endpoints targeted by a scan configuration.

Troubleshoot scan errors

Investigate endpoints with scan errors that have scan results older than two days and resolve the errors for each endpoint.

  1. From the Patch menu, go to Scan Management and then click Scan Errors.
  2. Review any scan errors.

For more information about how to find and resolve common scan errors, see How to Clear Those Pesky Patch Scan Errors: Troubleshooting Common Errors to Enable Successful Scans.

Offline CAB scans fail for Windows 7 and Windows Server 2008 R2

If offline CAB scans fail for Windows 7 and Windows Server 2008 R2 with the error:

Error creating Update Service Object - See C:\Windows\windowsupdate.log for more details

Check the Windows Update log for the following errors:

WARNING: Digital Signatures on file C:\Windows\SoftwareDistribution\ScanFile\f6f0081a-1e6e-4e64-a804-58cf334a1f48\Source.cab are not trusted: Error 0x800b0109

WARNING: Digital Signatures on file C:\Windows\SoftwareDistribution\ScanFile\f6f0081a-1e6e-4e64-a804-58cf334a1f48\Source.cab are not trusted: Error 0x800b0109

If you see these errors, then prerequisite patches might not be installed on the endpoints. To resolve this issue, use either Tanium Scan or a method outside of Patch to install updates for SHA2 signing and Extended Security Update on the endpoints. For more information, see Tanium Scan and Tanium Community: Enabling and Delivering Microsoft Extended Security Updates with Tanium.

Scans are not completed on Linux endpoints

Patch 2.3.5 supports Red Hat and CentOS Linux endpoints. Patch 2.4.3 also supports Oracle and Amazon Linux endpoints. Patch 3.2.160 also supports SUSE endpoints.If you are having difficulty getting scans to run on Linux endpoints:

  1. On the Patch Overview page, click Settings and then click Operating Systems to verify that the Enhanced Linux Support option is enabled.
  2. Verify that the Patch - Is Process Running sensor returns Yes for your Linux endpoints.
  3. Verify Contact Tanium Support to verify that the Tanium Server can reach the repomd.xml file by appending /repodata/repomd.xml to the configured baseurl value.
  4. Check the endpoint log at /opt/Tanium/TaniumClient/Tools/Patch/logs/patch-process.log for errors.

Red Hat Linux endpoints stuck in Waiting for Initial Scan status

If you configure a repository that includes both Red Hat Enterprise Linux 6 Server (RPMs) and Red Hat Enterprise Linux 7 Server (RPMs) endpoints, your targeted endpoints might appear to be stuck in the Waiting for Initial Scan status.

  1. Verify that each major operating system is targeted in repositories.
  2. Target Linux endpoints by major operating systems.

Linux repository snapshots issues

If you encounter issues creating or using snapshots, review the following solutions. For information about specific snapshot errors, see Reference: Common errors.

Warning about a failed snapshot

A warning message appears, to indicate that a snapshot failed. To modify the snapshot, click Manage Repository Snapshots. To remove the warning, click Dismiss warning.

Patch process is not running on Linux or macOS endpoints

Start the Patch process:

  1. In Interact, ask the Get Operating System from all machines question.
  2. From the grid, select the Linux operating systems.
  3. Click Deploy Action. Choose the Patch - Start Patch Process [Non-Windows] package.
  4. Click Preview and then click Deploy Action.

Check and update the Windows Update Agent

You can use Tanium to check which Windows Update Agent versions are installed on your Windows endpoints.

  1. In Interact, ask the Get File Version["C:\Windows\System32\wuaueng.dll"] from all machines question.
  2. Update any endpoints that have a version earlier than 6.1.0022.4. See the Microsoft article Updating the Windows Update Agent.

Monitor and troubleshoot Patch coverage

The following table lists contributing factors into why the Patch coverage metric might report endpoints as Needs Attention or Unsupported, and corrective actions you can make.

Contributing factor Corrective action
Gaps in Patch action group membership Ensure all operating systems that are supported by Patch are included in the Patch action group.
Gaps in scan configuration coverage

Review each scan configuration and which computer groups are targeted by each configuration.

Ensure that every endpoint that is supported by Patch is targeted by at least one scan configuration.

Scan frequency value is set too high Review each scan configuration to ensure that the Frequency value is set to less than three days for all scan configurations.
Scan windows are too restrictive Scan windows are optional. If you decide to use them, the Override option should be set to less than two days.
Unmitigated scan failures

Investigate endpoints with scan errors in scan results that are older than two days.

Remediate the error conditions on each endpoint. See Troubleshoot scan errors.

Patch process is not running Ensure that there are no conditions that could prevent the Patch process from running on endpoints that are included in the Patch action group.

Monitor and troubleshoot endpoints missing critical or important patches

The following table lists contributing factors into why the endpoints missing critical or important patches metric might be higher than expected, and corrective actions you can make.

Contributing factor Corrective action
Selective patching as a practice Avoid choosing specific patches based on vulnerability reports. Instead, use dynamic, rule-based patch lists. These lists should be cumulative. For example, do not create any rules that prevent patches that are older than a specific date from being included in a patch list.
Patches are deployed only for the current month Use dynamic, rule-based patch lists. These lists should be cumulative. For example, do not create any rules that prevent patches that are older than a specific date from being included in a patch list.
Having an n-1 patching policy

Include patches for the current month.

Expand endpoint diversity in patch testing groups to increase the changes of identifying newly-released problematic patches for deploying patches to production.

Not enforcing post-deployment restarts

Use the Restart option within deployments.

(Windows) Use the Notify User option and set the Deadline for restart value to less than a few days. To decrease both the endpoints missing critical or important patches and the mean time to patch metrics, the optimal value for this setting depends on your patching cycle. Successful customers find that setting the Deadline for restart value to less than three days is optimal.

Monitor and troubleshoot mean time to patch

The following table lists contributing factors into why the mean time to patch metric might be higher than expected, and corrective actions you can make.

Contributing factor Corrective action
Delayed testing cycle Begin the process of testing new monthly patches the day they are released, typically Patch Tuesday (second Tuesday of each month).
Delayed start of production cycle Avoid waiting longer than two weeks after a patch release to start patching production systems. The longer you wait to start patching production systems, the more aggressive the subsequent deployments need to be to complete the patching cycle in a reasonable time.
Staggering deployments to distribute the load on the network

Do not stagger deployments in an attempt to distribute the load on your network or Tanium. The more endpoints that are being patched simultaneously, the more efficient Tanium becomes with overall WAN usage.

For bandwidth-constrained locations, you can implement site throttles. For more information, see Tanium Console User Guide: Configure site throttles.

Staggering deployments to distribute the load on the Tanium Server or Patch Do not stagger deployments in an attempt to distribute the load on your network or Tanium.
Endpoints do not have enough time to install patches

Ensure that deployment windows are at least four hours and properly overlap with maintenance window times.

Ensure that maintenance windows are at least four hours long, repeat at least once each month, and properly overlap with deployment times and change control process timelines.

For deployments that are scheduled for the future, select the Download immediately option. If you find that endpoints are not completing patch installations within the specified windows, schedule the deployments even further in advance.

Attempting to minimize disruption to users with maintenance windows Use Tanium End-User Notifications instead of restrictive maintenance windows.
Not enforcing post-deployment restarts

Use the Restart option within deployments.

(Windows) Use the Notify User option and set the Deadline for restart value to less than a few days. To decrease both the endpoints missing critical or important patches and the mean time to patch metrics, the optimal value for this setting depends on your patching cycle. Successful customers find that setting the Deadline for restart value to less than three days is optimal.

Unmitigated installation failures

Identify endpoints with patch installation failures.

Investigate the specific error codes.

Remediate the conditions that caused the failures.

Endpoint health issues Ensure that endpoints have at least 5 GB of free space, a healthy Windows Update Agent, and an up-time of less than 90 days.

Remove Patch tools from endpoints

You can deploy an action to remove Patch tools from an endpoint or computer group. Separate actions are available for Windows and non-Windows endpoints.

  1. In Interact, target the endpoints from which you want to remove the tools. For example, ask a question that targets a specific operating system:
    Get Endpoint Configuration - Tools Status from all machines with Is Windows equals true
  2. In the results, select the row for Patch, drill down as necessary, and select the targets from which you want to remove Patch tools. For more information, see Tanium Interact User Guide: Drill Down.
  3. Click Deploy Action.
  4. For the Deployment Package, select Endpoint Configuration - Uninstall Tool [Windows] or Endpoint Configuration - Uninstall Tool [Non-Windows], depending on the endpoints you are targeting.
  5. For Tool Name, select Patch.

  6. (Optional) By default, after the tools are removed they cannot be reinstalled. To allow tools to be automatically reinstalled, clear the selection for Block reinstallation. Re-installation occurs almost immediately.

    If reinstallation is blocked, you must unblock it manually:

    • To allow Patch to reinstall tools, deploy the Endpoint Configuration - Unblock Tool [Windows] or Endpoint Configuration - Unblock Tool [Non-Windows] package (depending on the targeted endpoints).

    • If you reinstall tools manually, select Unblock Tool when you deploy the Endpoint Configuration - Reinstall Tool [Windows] or Endpoint Configuration - Reinstall Tool [Non-Windows] package.

  7. (Optional) To remove all Patch databases and logs from the endpoints, clear the selection for Soft uninstall.

    When you perform a hard uninstallation of some tools, such as Recorder or Index, the uninstallation also removes data that is associated with the tool from the endpoint. This data might include important historical or environmental data, such as recorded events (in the case of Recorder) or file indexes (in the case of Index). If data that you want to keep is associated with the tool, make sure you perform only a soft uninstallation of the tool. To help determine what data a tool stores on endpoints, go to https://docs.tanium.com/ and review the documentation for the tool or for the Tanium solution that installed it , and contact Tanium Support for additional help.

  8. (Optional) To also remove any tools that were dependencies of the Patch tools that are not dependencies for tools from other solutions, select Remove unreferenced dependencies.

  9. (Optional) In the Deployment Schedule section, configure a schedule for the action.

    If some target endpoints might be offline when you initially deploy the action, select Recurring Deployment and set a reissue interval.

  10. Click Show preview to continue.
  11. A results grid appears at the bottom of the page showing you the targeted endpoints for your action. If you are satisfied with the results, click Deploy Action.

If you have enabled Endpoint Configuration approval, tool removal must be approved in Endpoint Configuration before tools are removed from endpoints.

Uninstall Patch

If you need to uninstall Patch, first clean up the Patch artifacts on the endpoint and then uninstall Patch from the server.

  1. Clean up patch artifacts from the endpoints.
    1. Use Interact to target endpoints. To get a list of endpoints that have Patch, you can ask the Patch - Is Process Running question.
    2. Click Deploy Action. Choose the Patch - Clean Up Patch 2 Processes and Files package.
    3. Check the status of the action on the Actions > Action History page.
  2. Remove the Patch solution from the Tanium Module Server. From the Main menu, go to Administration > Configuration > Solutions.
    1. In the Patch section, click Uninstall and follow the process.
    2. Click Proceed with Uninstall.
      The uninstaller disables any actions.
    3. Return to the Solutions page and verify that the Import button is available for Patch.
      If the Patch module has not updated in the console, refresh your browser.

Restore the state of the Patch database

You can import the patch.db file to restore the Patch configuration.

  1. Stop the Patch service on the Tanium Module Server.
  2. Copy your patch.db file into the <Module Server>\services\patch-service\ directory, replacing the existing file.
  3. Restart the Patch service.
  4. In the Tanium Console, refresh the Patch workbench.
  5. Configure service account.
    Any existing data, including patch lists, deployments, and associated patches and actions appear in the Patch workbench.

    If a deployment scheduled action is missing, you might need to wait up to 5 minutes for it to show up.

Contact Tanium Support

To contact Tanium Support for help, sign in to https://support.tanium.com.