Indexing file systems

The TaniumCX binary is the framework for client extensions. TaniumCX loads components into memory for use by the client and Tanium solutions. Client extension processes are initiated and controlled by the TaniumClient.exe -m process.

Client extensions are installed by Tanium Endpoint Configuration and primarily exist on the file system within the TC\extensions folder. Client Extension logging is written to <Tanium Client>\Logs\extensions0.txt.

Two directories exist in the \extensions directory on each endpoint:

  • The first folder is the \extensions\config folder - which contains files to handle the manifest operations and maintains the tools database that contains all current tool versions and files
  • The second folder is the \extensions\core folder - which contains files to handle the I/O operations of Tanium solutions and facilitate communicate with the Tanium Server and Tanium Module Server.

Index performs two different kinds of work on files:

  • Work to generate properties about a file such as the time the file was created and the size of the file.
  • Work to generate properties about the contents of a file, such as the file hash, MIME type, and magic number.

Index calculates file content properties only when a file is in the scope of a solution that uses Index.

A scope defines the parameters of an Index scan, including a single scan origin that acts as the starting point of the scope. This can be a single directory, drive, or all drives. If a system has two drives, and you use the ‘all drives’ predefined origin, Index creates two scopes, one for each drive. As a result, two scans run in parallel.

Index resolves scope paths that are symbolic links at both the creation time of the scan object and when a scan starts. The registration reply includes the requested path to the physical path mapping. All paths reported from Index from a journal or database represent the physical path.

Index determines the scopes to which a given path belongs. When a scope is removed, Index checks the scope root for overlapping scans. If a scope no longer applies to any other scopes, Index triggers a removal of the root directory and quickly removes the data from the database.

Tanium solutions using Index data register a subscription with a domain and a name pair that must be unique to the subscription. This is the main identifier consumers use to configure Index. A subscription contains one or more scopes, which then define how Index should interact with a given directory. Scopes can and will overlap within a subscription or across subscriptions. The configuration parameters of one scope do not impact other scopes.

File system indexing occurs by "walking" the file system. A periodic crawl is necessary to ensure data correctness and integrity. Index cannot exclusively rely on the recorder for event data as it is entirely possible that a file has been modified offline. The walk frequency is controlled by the scope.

To walk a file system, the file tree must be complete. All scans start at the device root and walk subdirectories. When walking a file system, deleted files within a scope are detected. When entering a directory, Index checks the Index database to determine if the directory is listed. If the configured time has elapsed, Index compares the content in the database with what is presently detected on the filesystem. If it is time to enumerate, scope membership of the directory is determined. If no scopes include a particular directory, the walk stops.

If the scope membership specifies to open the file, the file modification time and last digested time values are compared with the database. If sufficient time has passed, or the mtime (modified time) differs, the digested work is performed and changes stored in the database.

Manage Index exclusions for Integrity Monitor, Reveal and Threat Response

To exclude files from indexing in Threat Response, see Create indexing exclusions.

To exclude files from indexing in Reveal versions before 1.18, use the Path Stem Exclusions or Path Filter Exclusions settings (from the Reveal Overview page, click Settings > Endpoint Configuration). To exclude files from indexing in Reveal versions 1.18 and later, use the Reveal Parse Exclusions by Regular Expression or Reveal Parse Exclusions by File Path settings in a Reveal profile.

To exclude files from indexing in Integrity Monitor versions 3.0 and later, use the Exclusions under Integrity Monitor > Watchlists > Watchlist Details > Path. As Integrity Monitor only watches file paths specified in the Watchlists, exclusions are only needed for files or directories in watched paths.

With Reveal 1.15 and later, exclusions defined in Reveal only apply to the Reveal Index Subscription.

With Integrity Monitor 3.0 and later, exclusions in Integrity Monitor only apply to the Integrity Monitor Index Subscription.

If a file is excluded in Threat Response, that file is excluded completely from Index, regardless of the settings in Reveal or Integrity Monitor.

Do not include a trailing slash in folder or directory exclusions. Index exclusions that end with trailing slashes indicate that all files in the directory are indexed, but Index does not add them to the Index database. Including a trailing slash in folder or directory exclusions can unnecessarily increase resource use on the endpoints.

Customize Index endpoint settings

Customize Index configuration settings to provide functionality and database parameters. Making changes to Index settings can cause performance impacts.

To change the value of a setting, use the TaniumClient config command:
./TaniumClient config set CX.index.<setting name> <value>

Alternatively, you can use Tanium Interact to issue a question, analyze the question results, and determine which endpoints require administrative action and deploy actions to those endpoints. To target endpoints, issue a question in Interact. In the Question Results grid, select the rows for the endpoints that require the action, and click Deploy Action. From the Deploy Action page, use the Deployment Package search box typeaheads to select packages. Select the Modify Tanium Client Setting or Modify Tanium Client Setting [Non-Windows] package. For the Windows package, REG_SZ is string, and REG_DWORD is int. For the Non-Windows package, the type is either string or numeric. For ValueName provide the fully-qualified name of the setting; for example, CX.index.MaxHashSizeMB. Provide a value. Configure a Deployment Schedule and Targeting Criteria. Click Deploy Action. For more information, see Deploying actions.

.

Setting Value Description

MaxHashSizeMB

INT

The maximum file size in MB to hash. (Default: 32)
FirstScanDistributeOverTimeMinutes INT The delay time (in minutes) after Index starts before starting the initial index scan. (Default: 1440)

FirstScanDistributeOverTimeTimeoutDays

INT

If a scan is overdue to start by more than this duration, reschedule the scan using the distribute over time logic. (Default: 7 days)
VolumeDiscoveryMaxDriveSizeGB INT By default Index does not scan volumes with used space >20TB. If index is requested to scan a large path explicitly, for example a high priority path, it scans that path regardless of size. (Default: 20k)
MaxZipSizeMB INT

Index extracts ZIP archives into memory. If the uncompressed size is larger than the provided value, Index does not extract and index the archive. (Default: 32 MB)

Do not set CX.index.MaxZipSizeMB = 0 without understanding the impact on other Tanium modules in the environment. For example, this causes Tanium Reveal and Tanium SBOM to not have visibiity into zip file contents. Similarly, setting CX.index.ZipRecursionLimit = 0 has the same effect.
ZipRecursionLimit INT Recursion limit for how deep to extract and index within ZIP archives. (Default: 10)