InfluxDB
InfluxDB Sink 连接器
描述
将数据写入 InfluxDB。
关键特性
选项
| 参数名 | 类型 | 必须 | 默认值 |
|---|---|---|---|
| url | string | 是 | - |
| database | string | 是 | |
| measurement | string | 是 | |
| username | string | 否 | - |
| password | string | 否 | - |
| key_time | string | 否 | processing time |
| key_tags | array | 否 | exclude field & key_time |
| batch_size | int | 否 | 1024 |
| max_retries | int | 否 | - |
| retry_backoff_multiplier_ms | int | 否 | - |
| connect_timeout_ms | long | 否 | 15000 |
| common-options | config | 否 | - |
url
连接到 influxDB 的 url,例如
http://influxdb-host:8086
database [string]
influxDB 数据库的名称
measurement [string]
influxDB measurement 的名称
username [string]
influxDB 用户名
password [string]
influxDB 用户密码
key_time [string]
在 SeaTunnelRow 中指定 influxDB measurement 时间戳的字段名。如果未指定,则使用处理时间作为时间戳
key_tags [array]
在 SeaTunnelRow 中指定 influxDB measurement 标签的字段名。
如果未指定,则包含所有字段作为 influxDB measurement 字段
batch_size [int]
对于批量写入,当缓冲区数量达到 batch_size 数量或时间达到 checkpoint.interval 时,数据将被刷新到 influxDB
max_retries [int]
刷新失败的重试次数
retry_backoff_multiplier_ms [int]
用作生成下一个退避延迟的乘数
max_retry_backoff_ms [int]
在尝试重新请求 influxDB 之前等待的时间量
connect_timeout_ms [long]
连接到 InfluxDB 的超时时间,以毫秒为单位
通用选项
Sink 插件通用参数,请参考 Sink 通用选项 详见
示例
sink {
InfluxDB {
url = "http://influxdb-host:8086"
database = "test"
measurement = "sink"
key_time = "time"
key_tags = ["label"]
batch_size = 1
}
}
多表
示例1
env {
parallelism = 1
job.mode = "STREAMING"
checkpoint.interval = 5000
}
source {
Mysql-CDC {
url = "jdbc:mysql://127.0.0.1:3306/seatunnel"
username = "root"
password = "******"
table-names = ["seatunnel.role","seatunnel.user","galileo.Bucket"]
}
}
transform {
}
sink {
InfluxDB {
url = "http://influxdb-host:8086"
database = "test"
measurement = "${table_name}_test"
}
}
变更日志
Change Log
| Change | Commit | Version |
|---|---|---|
| [Feature][Checkpoint] Add check script for source/sink state class serialVersionUID missing (#9118) | https://github.com/apache/seatunnel/commit/4f5adeb1c7 | 2.3.11 |
| [Improve] influxdb options (#8966) | https://github.com/apache/seatunnel/commit/9f498b8133 | 2.3.10 |
| [Improve] restruct connector common options (#8634) | https://github.com/apache/seatunnel/commit/f3499a6eeb | 2.3.10 |
| [Improve][dist]add shade check rule (#8136) | https://github.com/apache/seatunnel/commit/51ef800016 | 2.3.9 |
| [Feature][Restapi] Allow metrics information to be associated to logical plan nodes (#7786) | https://github.com/apache/seatunnel/commit/6b7c53d03c | 2.3.9 |
| [Improve] Improve some connectors prepare check error message (#7465) | https://github.com/apache/seatunnel/commit/6930a25edd | 2.3.8 |
| [Improve][Connector] Add multi-table sink option check (#7360) | https://github.com/apache/seatunnel/commit/2489f6446b | 2.3.7 |
| [Feature][Core] Support using upstream table placeholders in sink options and auto replacement (#7131) | https://github.com/apache/seatunnel/commit/c4ca74122c | 2.3.6 |
| Support multi-table sink feature for influxdb (#6278) | https://github.com/apache/seatunnel/commit/56f13e920d | 2.3.5 |
| [Improve][Zeta] Add classloader cache mode to fix metaspace leak (#6355) | https://github.com/apache/seatunnel/commit/9c3c2f183d | 2.3.5 |
| [Test][E2E] Add thread leak check for connector (#5773) | https://github.com/apache/seatunnel/commit/1f2f3fc5f0 | 2.3.4 |
| [BugFix][InfluxDBSource] Resolve invalid SQL in initColumnsIndex method caused by direct QUERY_LIMIT appendage with 'tz' function. (#4829) | https://github.com/apache/seatunnel/commit/deed9c62c3 | 2.3.4 |
| [Improve][Common] Introduce new error define rule (#5793) | https://github.com/apache/seatunnel/commit/9d1b2582b2 | 2.3.4 |
[Improve] Remove use SeaTunnelSink::getConsumedType method and mark it as deprecated (#5755) | https://github.com/apache/seatunnel/commit/8de7408100 | 2.3.4 |
| Support config column/primaryKey/constraintKey in schema (#5564) | https://github.com/apache/seatunnel/commit/eac76b4e50 | 2.3.4 |
| [Improve][Connector-V2] Remove scheduler in InfluxDB sink (#5271) | https://github.com/apache/seatunnel/commit/f459f500cb | 2.3.4 |
| [Improve][CheckStyle] Remove useless 'SuppressWarnings' annotation of checkstyle. (#5260) | https://github.com/apache/seatunnel/commit/51c0d709ba | 2.3.4 |
| Merge branch 'dev' into merge/cdc | https://github.com/apache/seatunnel/commit/4324ee1912 | 2.3.1 |
| [Improve][Project] Code format with spotless plugin. | https://github.com/apache/seatunnel/commit/423b583038 | 2.3.1 |
| [improve][api] Refactoring schema parse (#4157) | https://github.com/apache/seatunnel/commit/b2f573a13e | 2.3.1 |
| [Improve][build] Give the maven module a human readable name (#4114) | https://github.com/apache/seatunnel/commit/d7cd601051 | 2.3.1 |
| [Improve][Project] Code format with spotless plugin. (#4101) | https://github.com/apache/seatunnel/commit/a2ab166561 | 2.3.1 |
| [Improve][SourceConnector] Unifie InfluxDB source fields to schema (#3897) | https://github.com/apache/seatunnel/commit/85a984a64f | 2.3.1 |
| [Feature][Connector] add get source method to all source connector (#3846) | https://github.com/apache/seatunnel/commit/417178fb84 | 2.3.1 |
| [Feature][API & Connector & Doc] add parallelism and column projection interface (#3829) | https://github.com/apache/seatunnel/commit/b9164b8ba1 | 2.3.1 |
| [Hotfix][OptionRule] Fix option rule about all connectors (#3592) | https://github.com/apache/seatunnel/commit/226dc6a119 | 2.3.0 |
| [Improve][Connector-V2][Influxdb] Unified exception for influxdb source & sink connector (#3558) | https://github.com/apache/seatunnel/commit/4686f35d68 | 2.3.0 |
| [Feature][Connector][influx] Expose configurable options in influx db (#3392) | https://github.com/apache/seatunnel/commit/b247ff0aef | 2.3.0 |
| [Feature][Connector-V2] influxdb sink connector (#3174) | https://github.com/apache/seatunnel/commit/630e884791 | 2.3.0 |
| [Feature][Connector-V2] Add influxDB connector source (#2697) | https://github.com/apache/seatunnel/commit/1d70ea3084 | 2.3.0-beta |