Troubleshooting API Gateway
If API Gateway is not performing as expected, you might need to troubleshoot issues.
Collect logs
The information is saved as a ZIP file that you can download with your browser.
- From the API Gateway Overview page, click Help
, then the Troubleshooting tab.
- Click Download Support Package.
A tanium-api-gateway-support-[timestamp].zip file downloads to the local download directory. - Contact Tanium Support to determine the best option to send the ZIP file. For more information, see Contact Tanium Support.
Tanium API Gateway maintains logging information in the gateway-service.log file in the \Program Files\Tanium\Tanium Module Server\services\gateway-files\logs\ directory.
All-in-One deployment returns failed to initialize request error
If System User service is not installed and running, the API Gateway might return an HTTP error code 500
with the message Failed to initialize request
. For All-in-One deployments, before you install System User service as a required dependency for API Gateway, you must first add the 127.0.0.1
IPv4 address if you enabled IPv4, or the ::1
IPv6 address if you enabled IPv6, to the api_token_trusted_ip_address_list platform Server setting. Otherwise, the installation process does not complete.
All-in-One deployments are supported only for proof-of-concept (POC) demonstrations.
-
From the Main menu, go to Administration > Configuration > Platform Settings.
-
In the Name column, click api_token_trusted_ip_address_list.
-
You have the following options:
-
If you enabled IPv4, enter
127.0.0.1
in the Value field. -
If you enabled IPv6, enter
::1
in the Value field.
-
- Click Save.
Queries return unexpected results or errors
- The API Gateway service redirects queries and mutations to other Tanium solutions. If API Gateway returns unexpected results or errors, make sure that all prerequisites are installed at the minimum recommended version. For information, see API Gateway requirements.
- If all queries and mutations return a 502 Gateway Timeout error, make sure the Tanium System User Service and the Tanium API Gateway service are running on the Tanium Module Server.
- If you recently installed API Gateway or the System User Service, restart the Tanium Module Server.
- Queries and mutations that use the eid element require Interact 2.9 or later.
Request error handling
When requests cannot be fully satisfied, the API Gateway service gives the caller accurate, practical error information. The API Gateway service honors the GraphQL error specification with some custom extensions for greater specificity.
Syntax Validation
Before processing any portion of a request, the service parses its syntax. If the request is not valid GraphQL, or if it does not conform to the schema, the service returns a response with the syntax errors, including line and column positions whenever possible.
Examples of syntax errors include:
- Unbalanced parentheses, braces, brackets, and quotation marks
- References to fields that do not exist
The GraphQL error specification requires reporting these errors to the caller by line and column in the request.
Structural Validation
If the request passes syntax validation, the service evaluates the request for structural validity, which checks for errors not expressible with the GraphQL type system. Structural error examples include:
- Restrictions on type values, such as:
- A string may not be empty
- An integer must be positive
- Constraints on input objects, such as:
- Exactly one of two or more entries must be present
- Existence of values in a known set, such as:
- A named sensor must exist
- References to fields that cannot be satisfied in the current environment, such as:
- A sensor does not exist
The GraphQL extensions
error field for custom fields contains these errors.
Error Extensions
Errors that pertain to a specific field in the request are identified by a fieldPath
entry in the
extensions
object. Similar to the path
entry on the GraphQL error type, the fieldPath
consists
of a list of strings and integers which indicate keys in objects and offsets in lists, respectively.
The root of the fieldPath
is the request field that triggered the GraphQL error.
0
for the first item and increments for each additional item. For example, if a fieldPath
contains 5
, this refers to the sixth item in the list.
The extensions
object may also have these fields:
Extensions field | Description |
---|---|
code
|
an error code attributable to the field itself |
context
|
a map of contextual information that refines or specifies the error |
message
|
a short error explanation |
For example, with the following syntactically valid request:
{
endpoints {
edges {
node {
name
serialNumber
}
}
}
}
If the serial number is not available in the underlying data store, the API Gateway service returns a response including an extended error that the value does not exist:
{
"errors": [
{
"message": "Invalid request",
"path": ["endpoints"],
"extensions": {
"fieldPath": ["edges", "node", "serialNumber"],
"code": "validation_exists",
"message": "must refer to an existing sensor",
"context": {
"source": "tds"
}
}
}
]
}
Invalid argument errors contain an extended argumentErrors
entry. The value of that entry is a list of objects that may have the following entries:
Extensions field | Description |
---|---|
code
|
an error code attributable to the field itself |
context
|
a map of contextual information that refines or specifies the error |
path
|
the path to the argument error, relative to the field |
message
|
a short error explanation |
For example, with the following request that does not define a name
or path
value:
{
endpoints(filter: {filters: [{memberOf: {name: ""}}, {path:""}]}) {
edges {
node {
name
}
}
}
}
The API Gateway service returns a response including both errors stating a value is a required:
{
"errors": [
{
"message": "Invalid request",
"path": ["endpoints"],
"extensions": {
"argumentErrors": [
{
"code": "validation_required",
"path": ["filter", "filters", 0, "memberOf", "name"]
},
{
"code": "validation_required",
"path": ["filter", "filters", 1, "path"]
}
]
}
}
]
}
Extended errors may identify argument errors within sub-fields. For example, the following request specifies a non-existent sensor name:
{
endpoints {
edges {
node {
sensorReadings(sensors:[{name: "A Non-Existent Sensor"}]) {
columns {
values
}
}
}
}
}
}
The API Gateway service returns a response including the sub-field error that the value does not exist:
{
"errors": [
{
"message": "Invalid request",
"path": ["endpoints"],
"extensions": {
"fieldPath": ["edges", "node", "sensorReadings"],
"argumentErrors": [
{
"code": "validation_exists",
"path": ["sensors", 0, "name"]
}
]
}
}
]
}
Whenever possible, the response includes the valid portions of a query. For example, the following request references a valid sensor (Custom Tags) and a not valid sensor (A Non-existent Sensor):
{
endpoints {
edges {
node {
name
ipAddress
os {
generation
}
sensorReadings(sensors: [{name: "Custom Tags"}, {name: "A Non-existent Sensor"}]) {
columns {
sensor {
name
}
name
values
}
}
}
}
}
}
The response includes the error from A Non-Existent Sensor not being found, and all information that can otherwise be returned from the Custom Tags sensor:
{
"errors": [
{
"message": "sensors: (1: (name: must refer to an existing sensor.).).",
"path": [
"endpoints"
],
"extensions": {
"argumentErrors": [
{
"code": "validation_exists",
"context": {
"source": "tds"
},
"message": "must refer to an existing sensor",
"path": [
"sensors",
1,
"name"
]
}
],
"fieldPath": [
"edges",
"node",
"sensorReadings"
]
}
}
],
"data": {
"endpoints": {
"edges": [
{
"node": {
"name": "example-host",
"ipAddress": "192.0.2.10",
"os": {
"generation": "Windows 10"
},
"sensorReadings": {
"columns": [
{
"sensor": {
"name": "Custom Tags"
},
"name": "Custom Tags",
"values": [
"custom-tag-1",
"custom-tag-2"
]
}
]
}
}
}
]
}
}
}
Similar to lists in the fieldPath
referring to offsets in lists, if an error occurs on an item in a list, the error message
might refer to the list offset instead of the field name. In the above response, the message
refers to item 1
in the list; because list numbering starts at 0
, this points to the second sensor (A Non-existent Sensor).
Error codes
General error codes include:
Error code | Description |
---|---|
401 Unauthorized
|
If using an API token, you do not have permission to use the token for authentication, or the token is not valid or does not exist. |
403 Forbidden
|
You do not have the permissions required for your request. See the message for more information. |
502 Bad Gateway
|
The API Gateway service received an invalid response from the server. |
incompatible_solution
|
A required Tanium solution is not installed at the correct version. |
insufficient_permissions
|
The user account making the request does not have the proper permissions to access a resource. |
500 Internal Server Error
|
The response field cannot be resolved due to a server error. |
Syntax error codes include:
Error code | Description |
---|---|
GRAPHQL_PARSE_FAILED
|
The request contains unbalanced parentheses, braces, brackets, or quotation marks. See the message for more information. |
GRAPHQL_VALIDATION_FAILED
|
The request contains an unrecognized argument, or a required argument is not included. See the message for more information. |
INVALID_GRAPHQL
|
The request contains an unknown operation. See the message for more information. |
Structural error codes include:
Error code | Description |
---|---|
validation_allowed
|
The field is not allowed in this position. |
validation_date
|
The field must contain a valid RFC 3339 long date format (YYYY-MM-DD ). |
validation_exists
|
The field must refer to an existing entity. |
validation_in_invalid
|
The field must have a value from a specific set. |
validation_invalid_action_perform
|
The action operation must specify only one operation, specify package or operation , or a tag value contains whitespace characters or a number sign (# ). |
validation_invalid_dec_process_name
|
The request contains a process name that does not match the process ID. |
validation_invalid_field_filter
|
The field filter is not valid. |
validation_invalid_for_source
|
The source does not allow the request. |
validation_invalid_group
|
The request must provide name , id , or filter for the group. |
validation_invalid_ref
|
The reference must include either a name or ID. |
validation_invalid_use
|
The filter is not valid in its context. |
validation_integration_name
|
The integration name must contain a value that does not start with tanium- . |
validation_ipaddress
|
The field must contain an IPv4 address or IPv4 CIDR range. |
validation_key_unexpected
|
The field is not allowed. |
validation_min_greater_equal_than_required
|
The field must have a value greater than or equal to its documented minimum. |
validation_nil
|
The field must be absent. |
validation_nil_or_not_empty_required
|
The field must be absent or have a non-empty value. |
validation_not_nil_required
|
The field must have a value. |
validation_overdetermined
|
The field's object must not have inconsistent entries. |
validation_persona_name
|
The persona must be a persona assigned to the account that submitted the request. |
validation_positive_integer
|
The field must contain a positive integer. |
validation_required
|
The field must have a non-empty value. |
validation_timestamp
|
The field's value must be a valid RFC 3339 string. |
validation_unique
|
The field's value must be unique in its documented container. |
Issue: error message for enumerated value displays the value in the request
For enumerated type fields in requests, if you pass the enumerated value in quotation marks, the response error message appears to state that you must pass that value. For example, with the following request, any
(a boolean field) is set to "false" and visibility
(an enumerated type) is set to "ALL" (in quotation marks):
mutation createActionGroup {
actionGroupCreate(
input: {any: "false", computerGroups: [{name: "Example Computer Group"}], name: "Example Action Group", visibility: "ALL"}
) {
group {
id
}
error {
message
}
}
}
The response constrains validation error messages similar to the following because the validation engine parses "false" and "ALL" as non-enumerated values, and requires that they be passed in the request without quotation marks.
{
"errors": [
{
"message": "Boolean cannot represent a non boolean value: \"false\"",
"extensions": {
"code": "GRAPHQL_VALIDATION_FAILED"
}
},
{
"message": "Enum \"ActionGroupVisibility\" cannot represent non-enum value: \"ALL\". Did you mean the enum value \"ALL\"?",
"extensions": {
"code": "GRAPHQL_VALIDATION_FAILED"
}
}
]
}
Do one of the following:
-
Pass the value as a JSON variable definition.
-
Pass the value in your request without quotation marks.
-
If using the Query Explorer, enter an enumerated type field and a colon, press Shift + Space when your cursor is after the colon, then select an enumerated value.
Issue: ResourceExhausted error in response to valid query request
Query requests that retrieve a large amount of information in the response might return a ResourceExhausted
error, due to the size of the response in bytes being too large to return. For example, the following request specifies returning the first 99999 results, and requests multiple fields with the potential to return many results per field, increasing the overall size of the response:
query endpointsLargePagination {
endpoints(first: 99999) {
edges {
node {
id
eidFirstSeen
eidLastSeen
namespace
computerID
systemUUID
name
domainName
serialNumber
manufacturer
model
ipAddress
macAddresses
isVirtual
isEncrypted
chassisType
os {
name
language
}
services {
status
startupMode
}
installedApplications {
uninstallable
name
version
silentUninstallString
}
deployedSoftwarePackages {
id
name
applicability
gallery
}
compliance {
cveFindings {
cveYear
summary
severity
cveId
cvssScore
}
complianceFindings {
standard
standardVersion
state
id
ruleId
}
}
}
}
}
}
If this response size in bytes is too large, the response might include a ResourceExhausted
error code:
{
"errors": [
{
"message": "Error fetching endpoints: response error: rpc error: code = ResourceExhausted desc = grpc: received message larger than max (28156311 vs. 4194304)",
"extensions": {
"serviceName": "graphql-platform",
"code": "DOWNSTREAM_SERVICE_ERROR"
}
}
],
"data": {
"endpoints": null
}
}
Reduce the response size. You have the following options:
-
Reduce the number of results returned per page.
-
Reduce or otherwise constrain the information you request, especially if a given field might return an array for each result with multiple result values.
For the best results, due to the variability in size of Tanium deployments, retrieve 100 records per page as an initial test of your paginated query, and adjust the number of records returned per page based on performance and response time.
Uninstall API Gateway
If you need to uninstall API Gateway, perform the following steps.
Consult with Tanium Support before you uninstall or reinstall API Gateway.
- Sign in to the Tanium Console as a user with the Administrator role.
- From the Main menu, go to Administration > Configuration > Solutions.
- In the Content section, select the API Gateway row and click Uninstall.
- Review the summary and click Yes to proceed with the uninstallation.
- When prompted to confirm, enter your password.
The uninstall does not remove the API Gateway log from the Tanium Module Server. To remove the log after the uninstall completes, manually delete the \Program Files\Tanium\Tanium Module Server\services\gateway-files\ directory.
Contact Tanium Support
To contact Tanium Support for help, sign in to https://support.tanium.com.
Last updated: 5/30/2023 4:11 PM | Feedback