OpenSearch 输出插件
此插件通过 HTTP 将指标写入OpenSearch 实例。它支持 OpenSearch 1.x 和 2.x 版本，但不能保证与 1.x 的未来兼容性，而是将专注于 2.x 的支持。
请考虑为 1.x 版本使用现有的 Elasticsearch 插件。
引入于: Telegraf v1.29.0 标签: datastore, logging 操作系统支持: all
全局配置选项
插件支持其他全局和插件配置设置，用于修改指标、标签和字段，创建别名以及配置插件顺序等任务。更多详情请参阅 CONFIGURATION.md。
配置
# Configuration for OpenSearch to send metrics to.
[[outputs.opensearch]]
## URLs
## The full HTTP endpoint URL for your OpenSearch instance. Multiple URLs can
## be specified as part of the same cluster, but only one URLs is used to
## write during each interval.
urls = ["http://node1.os.example.com:9200"]
## Index Name
## Target index name for metrics (OpenSearch will create if it not exists).
## This is a Golang template (see https://pkg.go.dev/text/template)
## You can also specify
## metric name (`{{.Name}}`), tag value (`{{.Tag "tag_name"}}`), field value (`{{.Field "field_name"}}`)
## If the tag does not exist, the default tag value will be empty string "".
## the timestamp (`{{.Time.Format "xxxxxxxxx"}}`).
## For example: "telegraf-{{.Time.Format \"2006-01-02\"}}-{{.Tag \"host\"}}" would set it to telegraf-2023-07-27-HostName
index_name = ""
## Timeout
## OpenSearch client timeout
# timeout = "5s"
## Sniffer
## Set to true to ask OpenSearch a list of all cluster nodes,
## thus it is not necessary to list all nodes in the urls config option
# enable_sniffer = false
## GZIP Compression
## Set to true to enable gzip compression
# enable_gzip = false
## Health Check Interval
## Set the interval to check if the OpenSearch nodes are available
## Setting to "0s" will disable the health check (not recommended in production)
# health_check_interval = "10s"
## Set the timeout for periodic health checks.
# health_check_timeout = "1s"
## HTTP basic authentication details.
# username = ""
# password = ""
## HTTP bearer token authentication details
# auth_bearer_token = ""
## Optional TLS Config
## Set to true/false to enforce TLS being enabled/disabled. If not set,
## enable TLS only if any of the other options are specified.
# tls_enable =
## Trusted root certificates for server
# tls_ca = "/path/to/cafile"
## Used for TLS client certificate authentication
# tls_cert = "/path/to/certfile"
## Used for TLS client certificate authentication
# tls_key = "/path/to/keyfile"
## Send the specified TLS server name via SNI
# tls_server_name = "kubernetes.example.com"
## Use TLS but skip chain & host verification
# insecure_skip_verify = false
## Template Config
## Manage templates
## Set to true if you want telegraf to manage its index template.
## If enabled it will create a recommended index template for telegraf indexes
# manage_template = true
## Template Name
## The template name used for telegraf indexes
# template_name = "telegraf"
## Overwrite Templates
## Set to true if you want telegraf to overwrite an existing template
# overwrite_template = false
## Document ID
## If set to true a unique ID hash will be sent as
## sha256(concat(timestamp,measurement,series-hash)) string. It will enable
## data resend and update metric points avoiding duplicated metrics with
## different id's
# force_document_id = false
## Value Handling
## Specifies the handling of NaN and Inf values.
## This option can have the following values:
## none -- do not modify field-values (default); will produce an error
## if NaNs or infs are encountered
## drop -- drop fields containing NaNs or infs
## replace -- replace with the value in "float_replacement_value" (default: 0.0)
## NaNs and inf will be replaced with the given number, -inf with the negative of that number
# float_handling = "none"
# float_replacement_value = 0.0
## Pipeline Config
## To use a ingest pipeline, set this to the name of the pipeline you want to use.
# use_pipeline = "my_pipeline"
## Pipeline Name
## Additionally, you can specify a tag name using the notation (`{{.Tag "tag_name"}}`)
## which will be used as the pipeline name (e.g. "{{.Tag "os_pipeline"}}").
## If the tag does not exist, the default pipeline will be used as the pipeline.
## If no default pipeline is set, no pipeline is used for the metric.
# default_pipeline = ""必需参数
urls: 一个包含您 OpenSearch 实例的一个或多个节点的完整 HTTP URL 的列表。index_name: 指标的目标索引。您可以使用日期格式
例如：“telegraf-{{.Time.Format "2006-01-02"}}” 将其设置为“telegraf-2023-07-27”。您还可以指定指标名称 ({{ .Name }})、标签值 ({{ .Tag \"tag_name\" }}) 和字段值 ({{ .Field \"field_name\" }})。
如果标签不存在，默认标签值将是空字符串 ""
权限
如果您在 OpenSearch 集群中使用了身份验证，则需要创建一个账户并创建一个具有至少 Cluster Privileges 类别中 manage 角色的角色。否则，您的账户将无法连接到您的 OpenSearch 集群并向您的集群发送日志。之后，您需要为您的特定索引模式添加“create_index”和“write”权限。
OpenSearch 索引和模板
按时间范围划分的索引
此插件可以按时间范围管理索引，这在其他使用 OpenSearch 的工具中很常见。收集指标的时间戳将用于决定索引目标。有关此用法在 OpenSearch 上的更多信息，请查看 文档。
模板管理
索引模板在 OpenSearch 中用于定义索引的设置和映射以及字段应如何被分析。有关这方面工作原理的更多信息，请参阅 文档。
此插件可以创建一个工作模板供 telegraf 指标使用。它利用 OpenSearch 的动态模板功能来为标签和指标字段设置正确的类型。如果指定的模板已存在，除非您将此插件配置为这样做，否则它不会覆盖。因此，您可以在创建后根据需要自定义此模板。
telegraf 在 OpenSearch 2.x 上创建的索引模板示例
{
"telegraf-2022.10.02" : {
"aliases" : { },
"mappings" : {
"properties" : {
"@timestamp" : {
"type" : "date"
},
"disk" : {
"properties" : {
"free" : {
"type" : "long"
},
"inodes_free" : {
"type" : "long"
},
"inodes_total" : {
"type" : "long"
},
"inodes_used" : {
"type" : "long"
},
"total" : {
"type" : "long"
},
"used" : {
"type" : "long"
},
"used_percent" : {
"type" : "float"
}
}
},
"measurement_name" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"tag" : {
"properties" : {
"cpu" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"device" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"host" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"mode" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"path" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
}
}
}
}
},
"settings" : {
"index" : {
"creation_date" : "1664693522789",
"number_of_shards" : "1",
"number_of_replicas" : "1",
"uuid" : "TYugdmvsQfmxjzbGRJ8FIw",
"version" : {
"created" : "136247827"
},
"provided_name" : "telegraf-2022.10.02"
}
}
}
}事件示例
此插件将按以下方式格式化事件
{
"@timestamp": "2017-01-01T00:00:00+00:00",
"measurement_name": "cpu",
"cpu": {
"usage_guest": 0,
"usage_guest_nice": 0,
"usage_idle": 71.85413456197966,
"usage_iowait": 0.256805341656516,
"usage_irq": 0,
"usage_nice": 0,
"usage_softirq": 0.2054442732579466,
"usage_steal": 0,
"usage_system": 15.04879301548127,
"usage_user": 12.634822807288275
},
"tag": {
"cpu": "cpu-total",
"host": "opensearhhost",
"dc": "datacenter1"
}
}{
"@timestamp": "2017-01-01T00:00:00+00:00",
"measurement_name": "system",
"system": {
"load1": 0.78,
"load15": 0.8,
"load5": 0.8,
"n_cpus": 2,
"n_users": 2
},
"tag": {
"host": "opensearhhost",
"dc": "datacenter1"
}
}已知问题
大于 2^63 且小于 1e21 的整数值（或它们对应的负数值）将被 golang JSON 编码器以十进制格式编码，而 OpenSearch 动态字段映射对此并不完全支持。这会导致这些值的指标被丢弃，如果 telegraf 索引上尚未创建字段映射。如果发生这种情况，您将在 OpenSearch 端看到类似以下的异常
{
"error": {
"root_cause": [
{"type": "mapper_parsing_exception", "reason": "failed to parse"}
],
"type": "mapper_parsing_exception",
"reason": "failed to parse",
"caused_by": {
"type": "illegal_state_exception",
"reason": "No matching token for number_type [BIG_INTEGER]"
}
},
"status": 400
}当 OpenSearch 收到一个支持的 JSON 值后，将在 telegraf 索引上创建正确的字段映射，并且后续的插入将正常工作，因为字段映射将已存在。
此问题是由于 OpenSearch 尝试检测整数字段的方式以及 golang 在 JSON 中编码数字的方式引起的。目前没有明确的解决方法。
此页面是否有帮助？
感谢您的反馈！
支持和反馈
感谢您成为我们社区的一员！我们欢迎并鼓励您对 Telegraf 和本文档提出反馈和 bug 报告。要获取支持，请使用以下资源
具有年度合同或支持合同的客户可以 联系 InfluxData 支持。