Working with configuration compliance benchmarks and vulnerability sources

Before you can work with benchmarks, you must import them. See Download and import the CIS engine and Download and import the SCC scan engine for instructions on how to import or upgrade benchmarks in bulk.

After you import engine files, you can view imported benchmarks to verify that they have been imported. Click Benchmarks > Configuration Compliance to open the Configuration Compliance Benchmarks page or click Benchmarks > Vulnerability to open the Vulnerability Benchmarks page. You can view the corresponding imported benchmarks from these pages.

You must have the Comply Report Content Administrator role to read and write configuration compliance benchmarks and vulnerability sources. For more information about Comply roles, see User role requirements.

Configuration compliance benchmarks

Importing individual benchmarks and assigning categories

You can import benchmarks in Comply that have the following file formats:

  • Split XCCDF format: XCCDF file, OVAL file, CPE, and CPE-dictionary
  • Single SCAP 1.2 datastream single file
  • Multiple ZIP files containing split XCCDF files

Use categories and labels to group the benchmarks. You can filter the list of benchmarks on the Configuration Compliance Benchmarks page by category and label.

  1. On the Configuration Compliance Benchmarks page, click Import Benchmark to import configuration compliance benchmarks.
  2. Provide a Description for the benchmark.
  3. In the Category drop-down list, select an existing category or click Create new category... to create a category to assign to the benchmark.
    1. If you created a category, enter a name for your new category in the New Benchmark Category window, and click Save.

    When benchmarks are imported with an engine, such as CIS-CAT or SCC, they are automatically assigned the applicable category. By default, new benchmarks are assigned the Imported category.

  4. (Optional) Enter custom labels in the Labels field to describe the benchmark.
  5. Click Select Files and select the benchmark files.
  6. Click Import.


Filter benchmarks by category

You can filter benchmarks by category on the Configuration Compliance Benchmarks page by selecting a category from the Category drop-down list.

Change a benchmark category or label

  1. On the Configuration Compliance Benchmarks page, hover over a benchmark to show the Change Category/Labels button. Click this button.
  2. In the Benchmark Metadata window, select a different Category and, if needed, edit the Labels for the benchmark.
  3. Click OK.

Create a benchmark category from Settings

  1. From the Comply Home page, click Settings .
  2. On the Benchmark tab, click Create Category.
  3. In the New Benchmark Category window, enter a name for the category in the Category field
  4. Click Save.

Delete benchmark categories from Settings

  1. From the Comply Home page, click Settings .
  2. On the Benchmark tab, click Delete next to the benchmark category you want to delete.
  3. Click OK.

Filtering benchmarks

Use the Label Filter field at the top right of the Configuration Compliance Benchmarks page to filter the benchmarks. Begin entering text to see a list of available labels or click the X next to an existing label to remove it from the list of filters.

Benchmarks with a green verified label were tested and confirmed to work with Comply. Only verified benchmarks display by default. Benchmarks with a yellow unverified label were not tested with Comply. This label does not mean that the benchmark does not work with Comply. Benchmarks with a red unsupported label do not work with Comply.

Viewing configuration compliance benchmark profiles and creating reports

  1. Click Expand to see the details of a configuration compliance benchmark.
  2. In the Profiles section, click the Create Report link next to a benchmark profile to create a report for that profile.

    Click the Create Report button at the top to create a report for the first profile listed.

    The Create Configuration Compliance Report page opens. The Name, Platform, Benchmark, and Profile fields are automatically populated based on the benchmark and profile you selected. If needed, you can change these values.

  3. (Optional) Specify labels to describe the report in the Labels field.
  4. Select Computer Groups.

    Be sure to select the appropriate Platform (AIX, Linux, macOS, or Windows) and Computer Groups containing endpoints that align with the Platform for Comply to work correctly.

  5. If the platform was not set automatically, select a Platform.
  6. Select the Engine.

    The Engine field displays only when you have more than one engine installed.

  7. Select either Low or Normal from the Execution Priority drop-down list.

    If you select Low, the Comply scan process yields processor utilization to other processes running on the endpoint. If you select Normal, the scan process runs with the same priority as other processes on the endpoint.

    Selecting Low might increase the duration of the scan processes on endpoints with high processor utilization.

  8. (Optional) Select Start on and End on and complete the date and time values to limit the report to run only during a specific time period.
  9. (Optional) Select the Distribute Over option and enter values to run the report over minutes or hours. This value cannot be over four hours.
  10. In the Repeat field, select Interval, Using report age, or Never.
    • If you choose Interval, the Reissue every field displays, and you can specify how often the report runs.
    • If you choose Using report age, the Run when results are older than field displays, and you can specify how old you want the results to be before the report runs again. If a targeted endpoint comes online that has never run the report, the report runs as soon as the next age-check occurs. The age of results is checked every 3 hours unless you specify an age less than 3 hours. In this case, the age of results is checked every hour.
  11. Click Create & Deploy and enter your credentials. Action results will display.

Vulnerability sources

Default vulnerability sources

Expand Tanium Vulnerability Library to see the three vulnerability sources provided by Comply:

  • Tanium Vulnerability Library for Unix
  • Tanium Vulnerability Library for macOS
  • Tanium Vulnerability Library for Windows



Expand default vulnerability sources to view details, create a report, update them, or edit them. Click on the number of CVEs to see the full list of CVEs included in a benchmark. You can search the CVE list by using Filter by Text field.

Tanium maintains the Tanium Vulnerability Library daily. The new version of the Tanium Vulnerability Library is available for download by clicking Update in the Tanium Vulnerability Library section. To schedule automatic updates on a recurring schedule, click Edit .

Enabling recurring updates only updates definitions associated with the sources contained in the Tanium Vulnerability Library. If you created reports against one of the sources contained in the Tanium Vulnerability Library, these reports are not automatically updated. To update the vulnerabilities contained in a report, select Deploy Now from the Manage Report drop-down list next to a report on the Reports page. This action updates the corresponding source and then rebuilds the vulnerability definitions for the report. If the Start on field is not specified on the report's schedule, it runs immediately.

Search for CVEs

Enter one or more CVEs in the Search for CVEs field at the top of the Vulnerability Benchmarks page and click Search for CVEs . You can use a search to scan all vulnerability sources to identify which sources contain the specified CVEs.

Upload vulnerability source zip for air-gapped environments

If you are working in an air-gapped environment, you must specify that setting in Comply and then upload the air gap ZIP file.

  1. From the Comply Home page, click Settings .
  2. On the Application Settings tab, find the the is_airgapped setting and click Edit.
  3. In the Edit Setting window, enter true in the Value field.
  4. Click Save.
  5. If you edit the is_airgapped setting back to false, you must restart Comply for the Tanium Vulnerability Library (TVL) to update properly.

  6. After you specify that you are working in an air-gapped environment in the Comply settings, click Upload Airgap Zip on the Vulnerability Benchmarks page.
  7. Download the air gap ZIP file from the link indicated in the Upload TVL Airgap Zip window using a machine that can connect to the internet and save it on the air-gapped machine.
  8. Click Select File, select the air gap ZIP file from the location where you saved it on the air-gapped machine, and click Open.
  9. Click Upload.
  10. After your upload is complete, click Close on the Upload TVL Airgap Zip window. Allow approximately five minutes for Comply to update the vulnerability benchmarks. If you expand a vulnerability source, you will see the Type indicated as Local as well as a completed count of CVEs after the benchmarks are successfully updated from the uploaded air gap ZIP file.

Create a new vulnerability source

  1. Click Create Source on the Vulnerability Benchmarks page.
  2. In the Details section, provide a Name, Vendor, and Description.
  3. To schedule automatic updates, check Enable recurring updates.
  4. In the OVAL Definitions File section, choose either Remote or Upload for the Location.


  5. A remote source is a URL that points to an OVAL definition XML file and can be updated by clicking Update next to the benchmark on the Vulnerability Benchmarks page.

    Remote sources are best suited when OVAL content is updated periodically. Uploaded sources are best suited to air-gapped environments or when you would like to manually download and provide the source feed.

    This OVAL definition file from CIS provides access to all vulnerability definitions in their repository: https://oval.cisecurity.org/repository/download/5.11.1/all/oval.xml
  6. Enter the path for the Remote File or Select file for Upload as appropriate.
  7. Click Save.

Edit or delete a vulnerability source

On the Vulnerability Benchmarks page, click Edit to edit a vulnerability source or Delete to delete a vulnerability source.

Perform a vulnerability scan and create a report

  1. Click Expand to see the details of a vulnerability benchmark and view information about the associated XML file and operating systems. Click Create Report to create a new vulnerability report.
  2. For more information, see Create a Vulnerability Report .
  3. Click Create & Deploy to run the vulnerability scan.

Last updated: 9/17/2019 1:57 PM | Feedback