Working with configuration compliance benchmarks and vulnerability sources

Before you can work with benchmarks, you must import them. See Download and import the CIS engine and Download and import the SCC scan engine for instructions on how to import or upgrade benchmarks in bulk.

After importing engine files, you can view imported benchmarks to verify that they have been imported. Select Configuration Compliance or Vulnerability under Benchmarks to view each type of imported benchmark.

You must have the Comply Report Content Administrator role to read and write configuration compliance benchmarks and vulnerability sources. For more information about Comply roles, see User role requirements.

Configuration compliance benchmarks

Importing individual benchmarks and assigning categories

You can import benchmarks in Comply that have the following file formats:
• Split XCCDF format: XCCDF file, OVAL file, CPE, and CPE-dictionary
• Single SCAP 1.2 datastream single file
• Multiple zip files containing split XCCDF files

  1. On the Configuration Compliance page under Benchmarks, click Import Benchmark to import additional configuration compliance benchmarks.
  2. Provide a Description for the benchmark.
  3. Assign an existing category to the benchmark or create a new category to help organize your benchmarks. Select a category from the Category drop-down list or select Create new category... to create a new category to assign to the benchmark.
  4. Enter any custom tags in the Tags field to further describe and categorize the benchmark.
  5. When Benchmarks are imported with an engine, such as CIS-CAT or SCC, they are automatically assigned the applicable category. By default, new benchmarks are assigned the Imported Category.

  6. If you've chosen to create a new category, in the Create Category window, enter a name for your new category and click Save.

Filter benchmark categories

You can filter benchmarks by category on the Configuration Compliance Benchmarks page under Benchmarks by selecting a category in the Category drop-down list.

Change a benchmark category or tags

  1. On the Configuration Compliance Benchmarks page, hover over a benchmark and you will see the Change Category/Tags button.
  2. In the Benchmark Metadata window, select a different Category and edit Tags.
  3. Click OK.

Create a benchmark category from Settings

  1. At the top right of the Home page, click Settings .
  2. On the Benchmark Categories tab, click Create Category.
  3. In the New Benchmark Category window, enter a name for the category in the Category field and click Save.

Delete benchmark categories from Settings

  1. At the top right of the Home page, click Settings .
  2. On the Benchmark Categories tab, click Delete next to the benchmark category you want to delete.

Filtering benchmarks

Use the Tag Filter field at the top right of the Configuration Compliance Benchmarks page to filter the benchmarks. Begin entering text to see a list of available tags or click the X next to an existing tag to remove it from the list of filters.

The green supported tag indicates that Tanium has tested the benchmark. Only supported benchmarks are shown by default. Benchmarks indicated with a yellow unsupported tag are benchmarks that Tanium has not tested. This does not mean that the benchmark does not work; it has just not been tested by Tanium.

Viewing configuration compliance benchmark profiles and creating reports

  1. Click expand to see the details of a configuration compliance benchmark.
  2. Under the Profiles section, click Create Report next to a benchmark profile to create a report for that profile. Clicking Create Report at the top right creates a report for the first profile listed. The Name, Platform, Benchmark, and Profile fields are auto-populated based on the benchmark and profile you selected. These values can be changed.
  3. On the New Configuration Compliance Report page, enter a Name for the report in the Details section. You can also provide a Description.
  4. Select a Platform.
  5. Select the Engine. You will only see the Engine field if you have more than one engine installed.
  6. The Tanium Comply action group is created automatically by Comply and will be auto-populated in the Action Group field. All saved actions created by Comply will be created under this action group.

  7. Select Computer Groups.
  8. Be sure to select the appropriate platform (Windows, Linux, or OS X) and Computer Groups containing endpoints that align with the Platform for Comply to work correctly.

  9. Select either Low or Normal from the Execution Priority drop-down list. This will determine if other processes running on an endpoint take priority over running the report you are creating or if the report takes the same priority as other processes.
  10. Select Start at and End at and complete the date and time values to limit the report to run only during a specific time period.
  11. Select the Distribute over and enter values to run the report over minutes or hours. This value cannot be over four hours.
  12. Select None, Interval, or Report Result Age for the Repeat report execution by field.
  13. If you choose Interval, the Reissue every field will appear, and you must specify how often the report is run.
  14. If you choose Repeat Result Age, then the Run when results are older than field will appear, and you must specify how old you want the results to be before the report is run. If an endpoint comes online that has never run a report, it will run with this option selected. The age of results is checked either every hour or every 3 hours. If you specify an age less than 3 hours, the age of results will be checked every hour.
  15. Select the Benchmark and Profile from the drop-down lists in the Benchmarks section.
  16. Click + Add Additional Benchmark to add another benchmark or click Create & Deploy and enter your credentials. Action results will display.
  17. If you have Custom Checks or Custom ID Mappings, you can specify these in the Advanced section. See Deploying custom checks and using custom ID mapping for more information.

Vulnerability sources

Default vulnerability sources

By default, Comply provides three vulnerability sources:

  • Tanium Vulnerability Library for Unix
  • Tanium Vulnerability Library for macOS
  • Tanium Vulnerability Library for Windows

Expand default vulnerability sources to view details, create a report, update them, or edit them. Click on the number of CVEs to see the full list of CVEs included in a benchmark. You can search the CVE list using Filter by Text in the top right.




Tanium maintains these default vulnerability sources and makes updates to them daily as they become available. To schedule automatic updates to these vulnerability sources, check Enable recurring updates on the Edit Vulnerability Source page.



Enabling recurring updates only updates definitions associated with the source. If you have already created reports against the source, these reports do not get automatically updated. To update the vulnerabilities contained in a report, select Deploy Now from the Manage Report drop-down list next to a report on the Reports page. This will rebuild the report's vulnerability definitions after updating the corresponding source. If Start at is not specified on the report's schedule, it will run immediately.

Search for CVEs

Enter one or more CVEs in the Search for CVEs field at the top of the Vulnerability Benchmarks page and click find . A search can be used to scan all vulnerability sources to identify which sources contain the specific CVE(s).

Upload vulnerability source zip for air gapped environments

If you are working in an air gapped environment, you must specify that setting in Comply and then upload the airgap zip.

  1. At the top right of the Home page, click Settings .
  2. On the Application Settings tab, find the the is_airgapped setting and click Edit.
  3. In the Edit Setting window, enter true in the Value field.
  4. Click Save.
  5. If you edit the is_airgapped setting back to false, you must restart Comply in order for the Tanium Vulnerability Library (TVL) to update properly.

  6. Once you have specified that you are working in an air gapped environment in the Comply settings, click Upload Airgap Zip on the top right of the Vulnerability Benchmarks page.
  7. Download the airgap zip file from the link indicated in the Upload TVL Airgap Zip window using a machine that can connect to the internet and save it on the air gapped machine.
  8. Click Select File, select the airgap zip file from wherever you have saved it on the air gapped machine, and click Open.
  9. Click Upload.
  10. Once your upload is complete, click Close on the Upload TVL Airgap Zip window. Allow approximately five minutes for Comply to update the vulnerability benchmarks. If you expand a vulnerability source, you will see the Type indicated as Local as well as a completed count of CVEs once the benchmarks have been successfully updated from the uploaded airgap zip file.

Create a new vulnerability source

  1. Click Create Source on the top right of the Vulnerability Benchmarks page.
  2. In the Details section, provide a Name, Vendor, and Description.
  3. To schedule automatic updates, check Enable recurring updates.
  4. In the OVAL Definitions File section, choose either Remote or Upload for the Location.


  5. A remote source is a URL that points to an OVAL definition XML file and can be updated by simply clicking Update next to the benchmark on the Vulnerability Benchmarks page.

    Remote sources are best suited when OVAL content is updated periodically. Uploaded sources are best suited to air gapped environments or when you would like to manually download and provide the source feed.

    This OVAL definition file from CIS provides access to all vulnerability definitions in their repository: https://oval.cisecurity.org/repository/download/5.11.1/all/oval.xml
  6. Enter the path for the Remote File or Choose file for Upload as appropriate.
  7. Click Save.

Edit a vulnerability source

On the Vulnerability Benchmarks page, you can also select edit to edit a vulnerability source or delete to delete a vulnerability source.

Perform a vulnerability scan and create a report

  1. Click expand to see the details of a vulnerability benchmark to view information about the associated XML file and operating systems. Click Create Report at the top right to create a new vulnerability report.
  2. See Create a vulnerability report for more details.
  3. Click Create & Deploy to run the vulnerability scan.

Last updated: 12/4/2018 5:48 PM | Feedback