The Vector team is excited to announce version 0.47.0!
Release highlights:
opentelemetry source now supports metrics ingestion.window transform has been introduced which enables log noise reduction by filtering out events when the system is in a healthy state.mqtt source is now available, enabling ingestion from MQTT brokers.datadog_logs sink now supports a new conforms_as_agent option to format logs like the Datadog Agent, ensuring compatibility with reserved fields.opentelemetry source now supports metrics ingestion.window transform, a variant of ring buffer or backtrace logging implemented as a sliding window.
Allows for reduction of log volume by filtering out logs when the system is healthy, but preserving detailed
logs when they are most relevant.mqtt source enabling Vector to receive logs from a MQTT broker.vector graphwebsocket_server sink, allowing this component to cache latest received messages per client.convert_to of type ConversionUnit. By default, timing values in milliseconds (ms) are converted to seconds (s). Users can set convert_to to milliseconds to preserve the original millisecond values.address field is now available within VRL scripts when using the auth.strategy.custom authentication method.Sec-WebSocket-Protocol header in the websocket_server sink to better accommodate clients that require it.timeout config option to the healthcheck sink configuration. Previously it was hardcoded to 10 seconds across all components, but now it can be configured per component.Add wildcard_matching global config option to set wildcard matching mode for inputs. Relaxed mode allows configurations with wildcards that do not match any inputs to be accepted without causing an error.
Example config:
wildcard_matching: relaxed
sources:
stdin:
type: stdin
# note - no transforms
sinks:
stdout:
type: console
encoding:
codec: json
inputs:
- "runtime-added-transform-*"
framing.method = length_delimited in sinks for significantly improved performance with large (more than 10MB) batches.from and to parameters. There are no changes to the function signatures.file and files fields of remap transforms are now watched when --watch-config is enabled. Changes to these files automatically trigger a configuration reload, so there’s no need to restart Vector.amqp sink now supports setting the priority for messages. The value can be templated to an integer 0-255 (inclusive).deferred.max_age_secs and deferred.queue_url options to the aws_s3 and aws_sqs sinks, to automatically
route older event notifications to a separate queue, allowing prioritized processing of recent files.file source bug where known small files were not deleted after the specified remove_after_secs.region was missing from the STS authentication endpoint.aws_cloudwatch_logs sink to ~1MB.kubernetes source bug where use_apiserver_cache=true but there is no resourceVersion=0 parameter in list request. Per this issue, when resourceVersion =0 and !page_size.is_none inListParams, the parameter resourceVersion=0 will be ignored by kube-rs sdk. If no parameter resourceVersion passed to the apiserver, the apiserver will list pods from ETCD instead of in memory cache.Add an option in the datadog_logs sink to allow Vector to mutate the record to conform to the
protocol used by the Datadog Agent itself. To enable, use the conforms_as_agent option or have the
appropriate agent header (DD-PROTOCOL: agent-json) within the additional HTTP Headers list.
Any top-level fields that use Datadog-reserved keywords are moved into a new object named message. If message doesn’t exist, it is created first. For example:
{
"key1": "value1",
"key2": { "key2-1" : "value2" },
"message" : "Hello world",
... rest of reserved fields
}
will be modified to:
{
"message" : {
"message" : "Hello world",
"key1": "value1",
"key2": { "key2-1" : "value2" }
},
... rest of reserved fields
}
datadog_logs sink where the content of the log message is dropped when logs namespacing is enabled.Add a new extra_headers option to greptimedb_logs sink configuration to set additional headers for outgoing requests.
Change greptimedb_logs sink default content type to application/x-ndjson to match the default content type of greptimedb sink.
If you use the greptimedb version v0.12 or earlier, you need to set the content type to application/json in the sink configuration.
Example:
sinks:
greptime_logs:
type: greptimedb_logs
inputs: ["my_source_id"]
endpoint: "http://localhost:4000"
table: "demo_logs"
dbname: "public"
extra_headers:
x-source: vector
[sinks.greptime_logs]
type = "greptimedb_logs"
inputs = ["my_source_id"]
endpoint = "http://localhost:4000"
table = "demo_logs"
dbname = "public"
[sinks.greptime_logs.extra_headers]
x-source = "vector"
VRL was updated to v0.24.0. This includes the following changes:
The encode_gzip, decode_gzip, encode_zlib, and decode_zlib methods now use the zlib-rs backend.
which is much faster than the previous backend miniz_oxide.
The decode_base64, encode_base64, and decode_mime_q functions now use the SIMD backend.
which is faster than the previous backend.
Sign up to receive emails on the latest Vector content and new releases
Thank you for joining our Updates Newsletter