JSON Parser Transform

The Vector json_parser transform accepts and outputs log events, allowing you to parse a log field value as JSON.

Configuration

• Common
• Advanced

vector.toml

[transforms.my_transform_id]
  type = "json_parser" # required
  inputs = ["my-source-or-transform-id"] # required
  drop_field = true # optional, default
  drop_invalid = true # required
  field = "message" # optional, default

5 items

commonrequired

• boolcommonoptional

  drop_field

  If the specified field should be dropped (removed) after parsing. If parsing fails, the field will not be removed, irrespective of this setting.

  • Default: true
  • View examples
• boolcommonrequired

  drop_invalid

  If true events with invalid JSON will be dropped, otherwise the event will be kept and passed through. See Invalid JSON for more info.

  • No default
  • View examples
• stringcommonoptional

  field

  The log field to decode as JSON. Must be a string value type. See Field Notation Syntax and Invalid JSON for more info.

  • Default: "message"
  • View examples
• booloptional

  overwrite_target

  If target_field is set and the log contains a field of the same name as the target, it will only be overwritten if this is set to true.

  • Default: false
  • View examples
• stringoptional

  target_field

  If this setting is present, the parsed JSON will be inserted into the log as a sub-object with this name. If a field with the same name already exists, the parser will fail and produce an error. See Field Notation Syntax for more info.

  • No default
  • View examples

Examples

• Simple
• Wrapped

Given the following log event:

{
  "message": "{"key": "value"}"
}

You can parse the JSON with:

[transforms.json]
  inputs = ["<source_id>"]
  type   = "json_parser"
  field  = "message"

This would produce the following event as output:

{
  "key": "value"
}

By default, Vector drops fields after parsing them via the drop_field option.

How It Works

Chaining / Unwrapping

Please see the Examples section.

Complex Processing

If you encounter limitations with the json_parser transform then we recommend using a runtime transform. These transforms are designed for complex processing and give you the power of full programming runtime.

Environment Variables

Environment variables are supported through all of Vector's configuration. Simply add ${MY_ENV_VAR} in your Vector configuration file and the variable will be replaced before being evaluated.

You can learn more in the Environment Variables section.

Field Notation Syntax

The field and target_field options support Vector's field notation syntax, enabling access to root-level, nested, and array field values. For example:

vector.toml

[transforms.my_json_parser_transform_id]
  # ...
  field = "message"
  field = "parent.child"
  field = "array[0]"
  # ...

You can learn more about Vector's field notation in the field notation reference.

Invalid JSON

If the value for the specified field is not valid JSON you can control keeping or discarding the event with the drop_invalid option. Setting it to true will discard the event and drop it entirely. Setting it to false will keep the event and pass it through. Note that passing through the event could cause problems and violate assumptions about the structure of your event.

Merge Conflicts

Key conflicts

Any key present in the decoded JSON will override existing keys in the event.

Object conflicts

If the decoded JSON includes nested fields it will be deep merged into the event. For example, given the following event:

{
  "message": "{\"parent\": {\"child2\": \"value2\"}}",
  "parent": {
    "child1": "value1"
  }
}

Parsing the "message" field would result the following structure:

{
  "parent": {
    "child1": "value1",
    "child2": "value2"
  }
}

Notice that the parent.child1 key was preserved.