Working with benchmarks and vulnerability sources

You must have the Comply Report Content Administrator role to read and write configuration compliance benchmarks and vulnerability sources. For more information about Comply roles, see User role requirements.

Comply 2.4 and later includes several Tanium Certified benchmarks. These benchmarks have a tanium-certified label. You can use these benchmarks only with the Tanium Scan Engine (powered by JovalCM).

If you want to use other benchmarks or scan engines, you must import them. See Download and import the CIS engine and Download and import the SCC scan engine for instructions on how to import or upgrade benchmarks in bulk.

Configuration compliance benchmarks

Importing individual benchmarks and assigning categories

You can import benchmarks in Comply that have the following file formats:

  • Split XCCDF format: XCCDF file, OVAL file, CPE, and CPE-dictionary
  • Single SCAP 1.2 datastream single file
  • Multiple ZIP files containing split XCCDF files

Use categories and labels to group the benchmarks. You can filter the list of benchmarks on the Standards Compliance page by category and label.

  1. On the Standards Compliance page, click the Import Compliance Standards arrow to import configuration compliance benchmarks.
  2. Provide a Description for the benchmark.
  3. In the Category drop-down list, select an existing category or click Create new category... to create a category to assign to the benchmark.
    1. If you created a category, enter a name for your new category in the New Benchmark Category window, and click Save.

    When benchmarks are imported with an engine, such as CIS-CAT or SCC, they are automatically assigned the applicable category. By default, new benchmarks are assigned the Imported category.

  4. (Optional) Enter custom labels in the Labels field to describe the benchmark.
  5. Click Select Files and select the benchmark files.
  6. Click Import.


Filter standards by category

You can filter benchmarks by category on the Standards Compliance page by selecting a category from the Filter arrow and choosing from the Label drop-down list.

Change a Standards category or label

  1. On the Standards Compliance page, hover over a benchmark to show the Edit button. Click this button.
  2. In the Standard Metadata window, select a different Category and, if needed, edit the Labels for the benchmark.
  3. Click OK.

Create a standards category

  1. From the Comply Setup page, click Compliance.
  2. Click Create Profile.
  3. In the Standards Profile window, enter a title and a description.
  4. Select a Benchmark and click Create.

Delete Standards categories

  1. From the Comply Setup page, click Compliance.
  2. On the Custom Profiles tab, click Delete next to the category you want to delete.
  3. Click OK.

Filtering benchmarks

Use the Label Filter field at the top right of the Standards Compliance page to filter the benchmarks. Begin entering text to see a list of available labels or click the X next to an existing label to remove it from the list of filters.

Comply 2.4 and later includes several Tanium Certified benchmarks. These benchmarks have a tanium-certified label. You can use these benchmarks only with the Tanium Scan Engine (powered by JovalCM).

Benchmarks with a verified label were tested and confirmed to work with Comply. Only verified benchmarks display by default. Benchmarks with a yellow unverified label were not tested with Comply. This label does not mean that the benchmark does not work with Comply. Benchmarks with a red unsupported label do not work with Comply.

Viewing configuration compliance standard profiles and creating reports

  1. In Standards > Compliance, click Expand to see the details of a configuration compliance standard.
  2. In the Profiles section, click the Create Report link next to a benchmark profile to create a report for that profile.

    Click the Create Report button at the top to create a report for the first profile listed.

    The Create Configuration Compliance Report page opens. For the steps to create a configuration compliance report, see Create a configuration compliance report.

Vulnerability sources

Click Standards > Vulnerability to open the Vulnerability Standards page.

Default vulnerability sources

Expand Tanium Vulnerability Library to see the three vulnerability sources provided by Comply:

  • Tanium Vulnerability Library for Unix
  • Tanium Vulnerability Library for macOS
  • Tanium Vulnerability Library for Windows



Expand default vulnerability sources to view details, create a report, update them, or edit them. Click on the number of CVEs to see the full list of CVEs included in a standard. You can search the CVE list by using Filter by Name field.

Tanium maintains the Tanium Vulnerability Library daily. The new version of the Tanium Vulnerability Library is available for download by clicking Update Source in the Tanium Vulnerability Library section on the Vulnerability Standards page.

By default, the Tanium Vulnerability Library is automatically updated daily. To change this schedule, click Edit Source .

Keep the default schedule that updates the Tanium Vulnerability Library daily.

Comply checks approximately every 60 minutes to compare scheduled vulnerability reports against the most recent version of the Tanium Vulnerability Library. The report rebuilds if new definitions are available for any of the specified Vulnerability Content (Range of CVEs, CVSS Score, or Individual CVEs) and a report is scheduled to run.

Search for CVEs

Enter one or more CVEs in the Search CVEs field at the top of the Vulnerability Standards page and click Search for CVEs. You can use a search to scan all vulnerability sources to identify which sources contain the specified CVEs.

Upload the vulnerability source file for air-gapped environments

If you are working in an air-gapped environment, you must configure that setting in Comply and then upload the air gap ZIP file. For the steps to configure Comply for an air-gapped environment, see Configure Comply for an air-gapped environment.

  1. After you specify that you are working in an air-gapped environment in the Comply settings, click Standards > Vulnerability to open the Vulnerability Standards page.
  2. Click Upload Airgap Zip.
  3. Download the air gap ZIP file from the link indicated in the Upload TVL Airgap Zip window (https://content.tanium.com/files/published/tvl/Comply-Standards-Airgap-v1.zip) using a machine that can connect to the internet and save it on the air-gapped machine.
  4. Click Select File, select the Comply-TVL-Airgap-pkg.zip file from the location where you saved it on the air-gapped machine, and click Open.
  5. Click Upload.
  6. After your upload is complete, click Close on the Upload TVL Airgap Zip window. Allow approximately five minutes for Comply to update the vulnerability benchmarks. If you expand a vulnerability source, you will see the Type indicated as Local as well as a completed count of CVEs after the benchmarks are successfully updated from the uploaded air gap ZIP file.

Create a new vulnerability source

  1. Click Create Source on the Vulnerability Standards page.
  2. In the Details section, provide a Name, Vendor, and Description.
  3. To schedule automatic updates, check Enable recurring updates.
  4. In the OVAL Definitions File section, choose either Remote or Upload for the Location.


  5. A remote source is a URL that points to an OVAL definition XML file and can be updated by clicking Update next to the benchmark on the Vulnerability Benchmarks page.

    Remote sources are best suited when OVAL content is updated periodically. Uploaded sources are best suited to air-gapped environments or when you would like to manually download and provide the source feed.

    This OVAL definition file from CIS provides access to all vulnerability definitions in their repository: https://oval.cisecurity.org/repository/download/5.11.1/all/oval.xml
  6. Enter the path for the Remote File or Select file for Upload as appropriate.
  7. Click Save.

Edit or delete a vulnerability source

On the Vulnerability Standards page, click Edit to edit a vulnerability source or Delete to delete a vulnerability source.

Perform a vulnerability scan and create a report

  1. Click Expand to see the details of a vulnerability standard and view information about the associated XML file and operating systems. Click Create to create a new vulnerability report.
  2. For more information, see Create a vulnerability report .
  3. Click Create & Deploy to run the vulnerability scan.