Aggregations

Scanner supports some functions for basic aggregations.

Functions are invoked with the parentheses (), e.g. count(), although top-level parentheses are optional. E.g.:

# These are all OK
* | stats countdistinct(foo, bar)
* | stats(countdistinct(foo, bar))
* | countdistinct foo, bar
* | countdistinct(foo, bar)
# This is a parse error
* | stats countdistinct foo, bar

Function arguments can support either numeric or string values. Strings in function arguments may be specified as either bare words, or quoted strings. However, if a string begins with a number, it must be quoted.

# These are all OK
* | max "num_events"
* | max num_events
* | max "3d_objects"
# This is a parse error
* | max 3d_objects

Functions are chained onto text query filter clauses with the | (vertical pipe) operator, and can be attached to any legal query:

# get how many error responses occurred for each kubernetes pod
status_code >= 400 
| stats count() by kubernetes.container_name, kubernetes.pod_name

# get how many different emails there are in the whole dataset
* | countdistinct email

# get how many AWS API calls were made to S3 or DynamoDB and returned an error
(eventSource: "s3" or eventSource: "dynamodb") and errorCode: *
| stats count() as errorCount

# get the average, median, and 90th percentile S3 request counts by IAM user
userIdentity.type: "IAMUser" and eventSource: "s3.amazonaws.com"
| stats count() as numReqs, userIdentity.arn by userIdentity.arn
| stats avg(numReqs), percentile(50, numReqs), percentile(90, numReqs)