VictoriaMetrics vmanomaly 组件配置完全指南七大配置区块、数据流转与热重载实战【免费下载链接】VictoriaMetricsVictoriaMetrics: fast, cost-effective monitoring solution and time series database项目地址: https://gitcode.com/GitHub_Trending/vi/VictoriaMetricsVictoriaMetrics Anomaly Detectionvmanomaly是 VictoriaMetrics 生态中的智能异常检测服务通过 YAML 配置将「数据读取—模型训练—调度推理—结果回写」串成一条完整的自动化流水线。本文以 components/README.md 为骨架逐区块讲解settings、schedulers、models、reader、writer、monitoring、server七大配置区块的作用、必填/可选关系、最小可用配置、热重载机制与环境变量占位符并结合仓库中的 reader.md、models.md、scheduler.md 等子文档给出源码级的参数细节。读完本文你将能独立编写一份可运行的多对多 vmanomaly 配置并掌握配置热更新与敏感信息注入的最佳实践。七大配置区块总览与必填/可选关系vmanomaly的全部行为都由一份 YAML 配置驱动配置按职责划分为七个独立区块其中四个为必填、三个为可选配置区块是否必填职责Model(s) section必填定义在数据上运行的模型类型与超参数Reader section必填定义从哪个数据源、以何种查询读取数据Scheduler(s) section必填定义何时训练fit、何时推理inferWriter section必填定义异常分数等结果写回何处、如何命名Monitoring section可选开启 push/pull 两种自监控Settings section可选并行化、状态恢复、保留策略等全局设置Server section可选vmanomaly 自身的 HTTP 服务、REST API 与 UI有几点版本性约定值得注意以当前仓库文档标注为准自v1.7.2起服务会在启动时对配置做校验校验错误请查看容器日志各字段说明见上方各区块文档。自v1.13.0起组件类支持用短别名代替完整导入路径例如model.zscore.ZscoreModel可写为zscorereader.vm.VmReader可写为vmscheduler.periodic.PeriodicScheduler可写为periodic。本文所有示例均使用别名。自v1.13.0起支持preset预设模式见 Presets.md。此外Reader 与 Writer 还支持 多租户multitenancy通过tenant_id参数可以分别从不同租户读取、向不同租户写入适用于 VictoriaMetrics 集群版详见 Cluster-VictoriaMetrics.md 的 Multitenancy 章节。组件交互与数据流向下面这张图展示了vmanomaly各组件之间、以及与 VictoriaMetrics / VictoriaLogs / VictoriaTraces 数据源之间的交互关系来源于仓库中的 vmanomaly-components-diagram.md图中实线节点与箭头是必选的异常检测主链路其核心路径为config.yml → Scheduler → Reader → Model → WriterScheduler按时间表触发任务Reader向配置好的 VictoriaMetrics / VictoriaLogs / VictoriaTraces 数据源发起查询Model对查询结果执行拟合与推理Writer把产出的异常分数写回 VictoriaMetrics。图中虚线节点与箭头表示可选的**自监控Monitoring**集成——既可以把指标推送到 VictoriaMetrics也可以暴露/metrics端点供抓取。若配置了 Server 区块其本身也可作为自监控指标的发布端点此时monitoring.pull可以省略。最小完整配置示例多对多的模型—查询—调度映射下面的配置是文档给出的最小可用示例完整展示了当前版本支持的多模型 × 多查询 × 多调度器的多对多映射能力调度器负责何时跑模型声明跑哪个查询、用哪个调度器。为便于逐段讲解各区块注释已保留。settings: n_workers: 4 # number of workers to run models in parallel native_threads_per_worker: 0 # automatically divide container-aware CPU capacity across workers anomaly_score_outside_data_range: 5.0 # default anomaly score for anomalies outside expected data range restore_state: True # restore state from previous run, if available retention: # how long to keep stale models on disk/in memory ttl: 1d # time-to-live duration, if the model was not used for inference within this duration, it will be considered stale check_interval: 1h # how often to check for stale models and remove them # how and when to run the models is defined by schedulers schedulers: periodic_online: # alias class: periodic # scheduler class infer_every: 30s # how often to produce anomaly scores for new data scatter_infer_jobs: true # distribute infer jobs evenly across the infer interval to reduce synchronized bursts fit_every: 1000d # bootstrap-only schedule; use a finite cadence if accumulated state must be reset fit_window: 3d # how much historical data to use for fit stage start_from: 00:00 # align the bootstrap fit to midnight in the configured timezone tz: Europe/Kyiv # timezone to use for start_from periodic_online_weekly: class: periodic infer_every: 15m scatter_infer_jobs: true fit_every: 1000d # bootstrap-only schedule; use a finite cadence if accumulated state must be reset fit_window: 14d # if no start_from is specified, jobs will start immediately after service starts # what model types and with what hyperparams to run on your data models: zscore: # we can set up alias for model class: zscore_online # model class z_threshold: 3.5 decay: 0.99 # weight for data points value should be in (0, 1], 1 means to give equal weight to all data provide_series: [anomaly_score, y, yhat, yhat_upper] # what series to produce as output of the model queries: [host_network_receive_errors] # what queries to run particular model on schedulers: [periodic_online] # will be fit once, used for infer every 30s clip_predictions: True # clip predictions to expected data range, i.e. [0, inf] for this query host_network_receive_errors envelope_weekly: # we can set up alias for model class: temporal_envelope alpha: 0.005 # adapt the trend while using the bootstrap-only fit schedule loss_reactivity: 3 # allow new deviations to update the envelope provide_series: [anomaly_score, y, yhat, yhat_lower, yhat_upper] queries: [cpu_seconds_total] schedulers: [periodic_online_weekly] # fit on two weekly cycles, then update online every 15m anomaly_score_outside_data_range: 1.5 # override default anomaly score outside expected data range clip_predictions: True # clip predictions to expected data range, i.e. [0, inf] for this query cpu_seconds_total seasonalities: [hod_smooth, dow_smooth] # where to read data from reader: class: vm datasource_url: https://play.victoriametrics.com/ tenant_id: 0:0 sampling_period: 30s # what data resolution to fetch from VictoriaMetrics /query_range endpoint workers: 0 # automatically choose bounded datasource concurrency latency_offset: 1ms query_from_last_seen_timestamp: False tz: UTC # timezone to use for queries without explicit timezone offset: 0s # offset to apply to all queries, e.g. to account for data delays, can be overridden on per-query basis queries: # aliases to MetricsQL expressions cpu_seconds_total: expr: avg(rate(node_cpu_seconds_total[5m])) by (mode) # step: 30s # if not set, will be equal to reader-level sampling_period data_range: [0, inf] # query-level business policy from v1.30.2 detection_direction: above_expected # query-level from v1.30.2; detect spikes only min_dev_from_expected: [0.01, 0.01] # query-level from v1.30.2 host_network_receive_errors: expr: rate(node_network_receive_errs_total[3m]) / rate(node_network_receive_packets_total[3m]) step: 15m # here we override per-query sampling_period to request way less data from VM TSDB data_range: [0, inf] # query-level business policy from v1.30.2 detection_direction: above_expected # query-level from v1.30.2; detect spikes only min_dev_from_expected: 0.0 # query-level from v1.30.2; absolute-deviation filtering is disabled # where to write data to writer: datasource_url: http://victoriametrics:8428/ tenant_id: 0:0 # for VictoriaMetrics cluster, can support multitenant metric_format: __name__: $VAR for: $QUERY_KEY # enable self-monitoring in pull and/or push mode monitoring: # pull: # Enable /metrics endpoint. # addr: 0.0.0.0 # port: 8490 push: # Enable pushing self-monitoring metrics url: http://victoriametrics:8428 push_frequency: 15m # how often to push self-monitoring metrics # configure vmanomaly server and UI settings server: port: 8490 path_prefix: /vmanomaly # optional path prefix for all HTTP routes max_concurrent_tasks: 4 # maximum number of concurrent anomaly detection tasks processed by backend use_reader_connection_settings: True # if True, use readers datasource_url and credentials for UI requests to datasource uvicorn_config: # optional Uvicorn server configuration log_level: warning该配置的核心语义如下schedulers定义两个调度器periodic_online每 30 秒推理一次、以 3 天窗口做一次性引导拟合periodic_online_weekly每 15 分钟推理一次、以 14 天窗口拟合覆盖两个周周期。models定义两个模型zscore只跑在host_network_receive_errors查询上、由periodic_online驱动envelope_weeklyTemporal Envelope 在线模型只跑在cpu_seconds_total查询上、由periodic_online_weekly驱动并额外配置了anomaly_score_outside_data_range: 1.5覆盖全局默认值。reader定义两个 MetricsQL 查询host_network_receive_errors通过step: 15m覆盖了 reader 级sampling_period显著降低从 TSDB 读取的数据量。writer通过metric_format控制输出命名__name__: $VAR使输出指标名为anomaly_score、y、yhat等for: $QUERY_KEY添加查询别名标签。monitoring开启 push 自监控每 15 分钟推送一次指标。server开启 vmanomaly 自带 UI/API端口 8490路径前缀/vmanomaly。说明本示例以及 settings.md 中的示例使用fit_every: 1000d作为仅引导一次bootstrap-only的调度。这适用于自带遗忘/反应机制的在线模型例如zscore_online配合decay 1。如果需要显式丢弃过期历史则应改用有限拟合周期——每次 fit 都会用配置的fit_window重置在线模型状态。深入各区块Reader 与 per-query 参数Reader 是数据的入口。class: vmVmReader通过 MetricsQL 从 VictoriaMetrics/Prometheus 读取class: vlogsVLogsReaderv1.26.0 起通过 LogsQL 的statspipe 从 VictoriaLogs/VictoriaTraces 读取。详见 reader.md。自 v1.13.0 起queries支持**按查询per-query**配置子字段并覆盖 reader 级参数这正是上例中step: 15m生效的原理。常用子字段包括exprMetricsQL/PromQL 表达式即/query_range?query%s接受的内容step该查询返回数据点的频率覆盖 reader 级sampling_perioddata_rangev1.15.1合法数据范围。数据落在范围外会得到高异常分数1默认1.01可用模型级anomaly_score_outside_data_range调整模型预测落在范围外则异常分数为 0detection_directionv1.30.2both/above_expected/below_expected控制只检测向上还是向下的偏差min_dev_from_expectedv1.30.2绝对偏差阈值|y - yhat|小于该值时异常分数置 0可配置标量或双元素列表下/上两个方向min_rel_dev_from_expectedv1.30.2相对偏差阈值|y - yhat| / |yhat|小于该值时异常分数置 0max_points_per_queryv1.17.0拆分长fit_window查询的子区间上限避免单个查询超时tzv1.18.0、tenant_idv1.19.0、offsetv1.25.3分别覆盖时区、租户与查询时间偏移。reader 级还有workersv1.30.20表示按查询数与 CPU 自动选择有界并发、fetch_timeout/processing_timeoutv1.30.0分别控制数据源请求与结果后处理超时、series_processing_batch_sizev1.29.7高基数查询建议 4–16等参数。深入各区块Scheduler 的三种工作模式调度器决定多久跑一次、跑哪个时间范围的数据。class有三种取值periodicPeriodicScheduler生产环境常用。周期性地对新数据推理并按fit_every周期性重训模型以对抗数据漂移。核心参数为fit_window训练时间范围至少 1 秒、infer_every推理频率至少 1 秒、fit_every重训频率缺省等于infer_every、start_from/tzv1.18.5指定首次 fit 的启动时间与时区配合restore_state: true可避免重启后长时间空转、scatter_infer_jobsv1.29.7把推理任务均匀分散到推理间隔内降低突发负载。oneoffOneoffScheduler运行一次即退出适合测试或对历史数据一次性回填。通过fit_start_iso/fit_end_iso或fit_start_s/fit_end_s与infer_start_iso/infer_end_iso或infer_start_s/infer_end_s显式指定拟合与推理时间窗。backtestingBacktestingScheduler模拟周期性调度但在历史数据上只跑一次后退出用于评估模型在过去的实际表现。v1.22.1 推荐inference_only: true模式——由from/to定义仅用于推理的时间窗训练窗自动取每个推理段之前的fit_windowv1.28.0 的exact: true使在线模型按infer_every的小批量时序精确回放生产行为。自 v1.11.0 起配置区块需命名为schedulers复数旧的扁平scheduler写法会被隐式转换为默认别名default_scheduler并保留向后兼容。深入各区块Model 的类型、公共参数与输出模型是异常检测的核心。模型沿两个维度分类按输入处理方式**单变量univariate**模型对每条时间序列各训练一个实例**多变量multivariate**模型对一组对齐的时间序列共享一个实例可捕获跨序列的集体异常。按更新策略**离线offline**模型仅在fit时全量重训**在线online**模型v1.15.0在每个infer_every步长上做增量更新即使只有一个数据点也能更新参数显著降低数据源读取压力。内置模型包括auto自动调参、temporal_envelope复杂运营数据的首选在线模型支持趋势/日历/节假日/预测、mad_online基于 t-digest 的稳健中位数绝对偏差、quantile_online在线季节性分位数、zscore_online在线 Z 分数、rolling_quantile、prophet、isolation_forest_multivariate、holtwinters、std等。其中 Prophet、Isolation Forest、Holt-Winters 已标记为计划弃用文档建议新部署迁移到对应的 Temporal Envelope 形式。所有模型共享的公共参数包括queriesv1.10.0选择该模型使用的 reader 查询不写则默认使用 reader 中全部查询。schedulersv1.11.0选择驱动该模型的调度器不写则默认挂到全部调度器。provide_seriesv1.12.0限制回写的输出列如[anomaly_score]timestamp列会被隐式加入。scalev1.20.0 支持双向以[scale_lower, scale_upper]分别缩放下/上置信区间宽度。clip_predictionsv1.20.0把yhat系列裁剪到data_range内。anomaly_score_outside_data_rangev1.20.0覆盖数据越界时的异常分数默认 1.01。decayv1.23.0仅在线模型指数遗忘因子取值(0.0, 1.0]1.0表示不衰减。groupbyv1.13.0仅多变量模型按标签值分组每组各训一个独立多变量模型。注意data_range、detection_direction、min_dev_from_expected、min_rel_dev_from_expected在模型级配置已自v1.30.2起弃用应迁移到reader.queries.alias下的查询级策略查询级显式值具有权威性模型级旧值仅在查询未定义时作为本地回退。vmanomaly的标准输出指标为anomaly_score主指标0–1 视为正常大于 1 判定异常且跨模型归一化、yhat预测期望值、yhat_lower/yhat_upper预测下/上界、y原始值。若infer收到 NaN 或无穷大输入对应anomaly_score为 NaN。深入各区块Writer 的指标格式化与多租户Writer 负责把模型输出写回 VictoriaMetrics其metric_format有两个必填键__name__必须包含$VAR占位符用于区分输出指标类型例如__name__: vmanomaly_$VAR会生成vmanomaly_anomaly_score、vmanomaly_yhat_lower等for通常填$QUERY_KEY为每条输出附加查询别名标签。其余键为用户自定义标签输入查询自带的标签如cpu1, deviceeth0, instancenode-exporter:9100会被原样继承到输出指标上。此外 writer 还支持 mTLSverify_tls/tls_cert_file/tls_key_file、BasicAuth、bearer token以及 v1.30.3 起的batch_max_series/batch_max_bytes/metric_prefix_cache_max_entries批量写入调优参数。多租户场景下tenant_id支持multitenant端点跨租户写入但需要注意聚合查询可能丢失vm_account_id路由标签而回落到默认租户0:0会打印警告。深入各区块Settings、Monitoring 与 ServerSettingssettings.md控制服务级行为n_workers与native_threads_per_workerv1.30.2控制进程级并行与数值库线程数restore_statev1.24.0使服务有状态重启后从$VMANOMALY_MODEL_DUMPS_DIR/vmanomaly.db恢复模型与调度器状态需开启磁盘模式且模型签名变化时自动重训retentionv1.28.1以ttlcheck_interval清理长期运行中累积的陈旧模型实例logger_levelsv1.25.3支持按组件前缀设置日志级别并支持热更新。Monitoringmonitoring.md提供 push 与 pull 两种自监控push可配置url、push_frequency默认 15m置空字符串可禁用定时推送仅保留 fit/infer 阶段推送、extra_labels等pull配置addr/port暴露/metrics。服务会产出vmanomaly_reader_*、vmanomaly_model_*、vmanomaly_writer_*、vmanomaly_config_reload*、vmanomaly_scheduler_*等系列自监控指标。Serverserver.md负责 REST API、/metrics端点与 Web UIport默认 8490、path_prefix可为所有路由加前缀如/vmanomaly后 UI 地址为http://localhost:8490/vmanomaly/vmui/、max_concurrent_tasks默认 2、ui_default_state可指定 UI 默认状态、use_reader_connection_settingsv1.29.2让 UI 复用 reader 的数据源连接凭据。v1.30.0 还提供了GET /api/v1/timeseries/characteristics序列特征分析与POST/GET/DELETE /api/v1/autotune/tasks异步共享调参任务端点。配置热重载Hot Reload自v1.25.0起vmanomaly支持无需重启进程地热重载配置文件。启用方式是在命令行加--watch参数详见 QuickStart.md 的 Command-line arguments 一节usage: vmanomaly.py [--license STRING | --licenseFile PATH] [--license.forceOffline] [--loggerLevel {DEBUG,INFO,WARNING,ERROR,FATAL}] [--watch] [-configCheckInterval DURATION] [--dryRun] [--outputSpec PATH] config [config ...]热重载最好与有状态服务stateful service配合使用——通过restore_state保留模型与调度器状态重启后无需重新训练模型、重新初始化调度器、重新读取数据最大化复用已有成果。自监控指标vmanomaly_config_reload_enabled在启用热重载时为1否则为0。[!WARNING] 自v1.29.5起基于文件系统事件的旧式热重载已被弃用改为基于内容轮询的方式原因是 Kubernetes ConfigMap 符号链接轮换等场景下事件投递不可靠。如果此前使用的是文件系统事件式热重载请改用--watch标志并按要求配置-configCheckInterval。热重载的工作原理服务按-configCheckInterval默认30s轮询被监听的.yml/.yaml文件内容v1.29.5。当检测到内容变化时会先等待防抖窗口然后重建全局配置并重新初始化各组件。vmanomaly_config_reloads_total指标会以statussuccess或statusfailure递增校验失败也会记录到日志。关键稳定性保证是如果重载失败服务会记录失败原因日志并继续沿用上一次有效的配置运行直到某次重载成功。这意味着新配置即使有错误服务也不会中断而是保持最后一份有效配置继续工作。在分片sharded部署中每次全局配置变更都会重新计算当前分片的归属。自 v1.30.4 起没有可运行任务的分片会保持存活但空闲后续重载若分配了任务它会在不重启进程的情况下恢复兼容的模型状态、创建调度器并开始执行。热重载使用的分片拓扑来自进程启动时的环境变量——变更分片数量、成员索引、副本因子或分配策略都需要编排层滚动升级或重启进程。分配策略方面ROUND_ROBIN在增删实体时可能移动规范有序子配置的后缀RENDEZVOUS在分片集合不变时保持无关分配的稳定性见 Scaling-vmanomaly.md 的分配策略指引。热重载示例假设服务以config.yaml启动内容如下settings: n_workers: 4 # number of workers to run models in parallel anomaly_score_outside_data_range: 5.0 # default anomaly score for anomalies outside expected data range restore_state: True # restore state from previous run, if available schedulers: periodic: class: periodic infer_every: 30s fit_every: 1000d # bootstrap-only schedule; use a finite cadence if accumulated state must be reset fit_window: 24h reader: datasource_url: https://play.victoriametrics.com/ tenant_id: 0:0 class: vm sampling_period: 30s queries: cpu_seconds_total: expr: avg(rate(node_cpu_seconds_total[5m])) by (mode) data_range: [0, inf] # step: 30s # if not set, will be equal to reader-level sampling_period host_network_receive_errors: expr: rate(node_network_receive_errs_total[3m]) / rate(node_network_receive_packets_total[3m]) step: 15s data_range: [0, inf] models: zscore: class: zscore_online z_threshold: 3.5 decay: 0.99 # gives more weight to recent data points, value should be in (0, 1], 1 means to give equal weight to all data provide_series: [anomaly_score] # if queries are not specified, all queries from reader will be used # if schedulers are not specified, all schedulers will be used writer: datasource_url: http://victoriametrics:8428/ tenant_id: 0:0 monitoring: push: url: http://victoriametrics:8428 push_frequency: 15m假设服务启动 15 分钟后reader.queries中cpu_seconds_total的查询表达式与频率发生了变化# ... (rest of the config remains unchanged) reader: # ... (rest of the reader config remains unchanged) queries: cpu_seconds_total: expr: avg(rate(node_cpu_seconds_total[10m])) by (mode) # changed lookback period data_range: [0, inf] step: 60s # changed step # ... (rest of the config remains unchanged)保存改动后热重载会自动检测config.yaml的内容变化并尝试重载。由于改动有效服务会记录成功日志并以statussuccess递增vmanomaly_config_reloads_total。重载后的实际效果是按需复用、只重训受影响的部分所有在host_network_receive_errors上训练的zscore_online模型实例仍然有效可继续直接对新数据点推理直到下一个fit_every触发所有在cpu_seconds_total上训练的zscore_online模型实例因查询表达式与频率变化而失效会以新的查询表达式与频率重新训练。环境变量占位符安全注入敏感配置自v1.25.0起配置文件中可以直接引用环境变量语法为标量字符串占位符%{ENV_NAME}。这对管理 API Key、数据库凭据等敏感信息特别有用——敏感值不必硬编码进配置文件而是由部署环境注入。例如设置环境变量VMANOMALY_URLhttp://localhost:8428后可在 reader 区块中写datasource_url: %{VMANOMALY_URL}启动时即被替换为实际值。注意如果引用的环境变量未设置或拼写有误占位符不会被替换可能导致配置校验失败或端点探测失败。因此建议在启动服务前确保所有必需的环境变量都已就绪。环境变量示例reader: class: vm datasource_url: %{VMANOMALY_URL} # will be replaced with the value of VMANOMALY_URL environment variable tenant_id: %{VMANOMALY_TENANT_ID} # will be replaced with the value of VMANOMALY_TENANT_ID environment variable bearer_token: %{VMANOMALY_BEARER_TOKEN} # will be replaced with the value of VMANOMALY_BEARER_TOKEN environment variable sampling_period: 30s writer: datasource_url: %{VMANOMALY_URL} # will be replaced with the value of VMANOMALY_URL environment variable tenant_id: %{VMANOMALY_TENANT_ID} # will be replaced with the value of VMANOMALY_TENANT_ID environment variable bearer_token: %{VMANOMALY_BEARER_TOKEN} # will be replaced with the value of VMANOMALY_BEARER_TOKEN environment variable # other config sections ...上例中同一组VMANOMALY_URL、VMANOMALY_TENANT_ID、VMANOMALY_BEARER_TOKEN被 reader 与 writer 同时引用既避免了重复书写也保证了读写两端凭据一致。小结与延伸阅读一份可用的vmanomaly配置可以归纳为四句话reader决定读什么、schedulers决定何时跑、models决定怎么判、writer决定写哪去再按需叠加settings并行/状态/保留、monitoring自监控与serverUI/API。在此基础上--watch-configCheckInterval提供滚动更新能力%{ENV_NAME}占位符保证敏感信息可安全注入而--dryRun可在不启动服务、不需要 license 的前提下提前校验整份配置含多 YAML 合并与 schema 校验是上线前必做的检查步骤。若想继续深入建议依次阅读仓库中的components/reader.md——VmReader 与 VLogsReader 的全部参数、per-query 参数与 MetricsQL/LogsQL 查询示例components/models.md——内置模型矩阵、公共参数、模型输出与自定义模型指南components/scheduler.md——periodic / oneoff / backtesting 三种调度器的完整参数components/settings.md——并行化、状态恢复与保留策略components/monitoring.md 与 components/server.md——自监控指标与 REST API/UIQuickStart.md——命令行参数、Docker 部署与 license 配置。【免费下载链接】VictoriaMetricsVictoriaMetrics: fast, cost-effective monitoring solution and time series database项目地址: https://gitcode.com/GitHub_Trending/vi/VictoriaMetrics创作声明：本文部分内容由AI辅助生成（AIGC），仅供参考 VictoriaMetrics vmanomaly 组件配置完全指南：七大配置区块、数据流转与热重载实战
VictoriaMetrics vmanomaly 组件配置完全指南七大配置区块、数据流转与热重载实战【免费下载链接】VictoriaMetricsVictoriaMetrics: fast, cost-effective monitoring solution and time series database项目地址: https://gitcode.com/GitHub_Trending/vi/VictoriaMetricsVictoriaMetrics Anomaly Detectionvmanomaly是 VictoriaMetrics 生态中的智能异常检测服务通过 YAML 配置将「数据读取—模型训练—调度推理—结果回写」串成一条完整的自动化流水线。本文以 components/README.md 为骨架逐区块讲解settings、schedulers、models、reader、writer、monitoring、server七大配置区块的作用、必填/可选关系、最小可用配置、热重载机制与环境变量占位符并结合仓库中的 reader.md、models.md、scheduler.md 等子文档给出源码级的参数细节。读完本文你将能独立编写一份可运行的多对多 vmanomaly 配置并掌握配置热更新与敏感信息注入的最佳实践。七大配置区块总览与必填/可选关系vmanomaly的全部行为都由一份 YAML 配置驱动配置按职责划分为七个独立区块其中四个为必填、三个为可选配置区块是否必填职责Model(s) section必填定义在数据上运行的模型类型与超参数Reader section必填定义从哪个数据源、以何种查询读取数据Scheduler(s) section必填定义何时训练fit、何时推理inferWriter section必填定义异常分数等结果写回何处、如何命名Monitoring section可选开启 push/pull 两种自监控Settings section可选并行化、状态恢复、保留策略等全局设置Server section可选vmanomaly 自身的 HTTP 服务、REST API 与 UI有几点版本性约定值得注意以当前仓库文档标注为准自v1.7.2起服务会在启动时对配置做校验校验错误请查看容器日志各字段说明见上方各区块文档。自v1.13.0起组件类支持用短别名代替完整导入路径例如model.zscore.ZscoreModel可写为zscorereader.vm.VmReader可写为vmscheduler.periodic.PeriodicScheduler可写为periodic。本文所有示例均使用别名。自v1.13.0起支持preset预设模式见 Presets.md。此外Reader 与 Writer 还支持 多租户multitenancy通过tenant_id参数可以分别从不同租户读取、向不同租户写入适用于 VictoriaMetrics 集群版详见 Cluster-VictoriaMetrics.md 的 Multitenancy 章节。组件交互与数据流向下面这张图展示了vmanomaly各组件之间、以及与 VictoriaMetrics / VictoriaLogs / VictoriaTraces 数据源之间的交互关系来源于仓库中的 vmanomaly-components-diagram.md图中实线节点与箭头是必选的异常检测主链路其核心路径为config.yml → Scheduler → Reader → Model → WriterScheduler按时间表触发任务Reader向配置好的 VictoriaMetrics / VictoriaLogs / VictoriaTraces 数据源发起查询Model对查询结果执行拟合与推理Writer把产出的异常分数写回 VictoriaMetrics。图中虚线节点与箭头表示可选的**自监控Monitoring**集成——既可以把指标推送到 VictoriaMetrics也可以暴露/metrics端点供抓取。若配置了 Server 区块其本身也可作为自监控指标的发布端点此时monitoring.pull可以省略。最小完整配置示例多对多的模型—查询—调度映射下面的配置是文档给出的最小可用示例完整展示了当前版本支持的多模型 × 多查询 × 多调度器的多对多映射能力调度器负责何时跑模型声明跑哪个查询、用哪个调度器。为便于逐段讲解各区块注释已保留。settings: n_workers: 4 # number of workers to run models in parallel native_threads_per_worker: 0 # automatically divide container-aware CPU capacity across workers anomaly_score_outside_data_range: 5.0 # default anomaly score for anomalies outside expected data range restore_state: True # restore state from previous run, if available retention: # how long to keep stale models on disk/in memory ttl: 1d # time-to-live duration, if the model was not used for inference within this duration, it will be considered stale check_interval: 1h # how often to check for stale models and remove them # how and when to run the models is defined by schedulers schedulers: periodic_online: # alias class: periodic # scheduler class infer_every: 30s # how often to produce anomaly scores for new data scatter_infer_jobs: true # distribute infer jobs evenly across the infer interval to reduce synchronized bursts fit_every: 1000d # bootstrap-only schedule; use a finite cadence if accumulated state must be reset fit_window: 3d # how much historical data to use for fit stage start_from: 00:00 # align the bootstrap fit to midnight in the configured timezone tz: Europe/Kyiv # timezone to use for start_from periodic_online_weekly: class: periodic infer_every: 15m scatter_infer_jobs: true fit_every: 1000d # bootstrap-only schedule; use a finite cadence if accumulated state must be reset fit_window: 14d # if no start_from is specified, jobs will start immediately after service starts # what model types and with what hyperparams to run on your data models: zscore: # we can set up alias for model class: zscore_online # model class z_threshold: 3.5 decay: 0.99 # weight for data points value should be in (0, 1], 1 means to give equal weight to all data provide_series: [anomaly_score, y, yhat, yhat_upper] # what series to produce as output of the model queries: [host_network_receive_errors] # what queries to run particular model on schedulers: [periodic_online] # will be fit once, used for infer every 30s clip_predictions: True # clip predictions to expected data range, i.e. [0, inf] for this query host_network_receive_errors envelope_weekly: # we can set up alias for model class: temporal_envelope alpha: 0.005 # adapt the trend while using the bootstrap-only fit schedule loss_reactivity: 3 # allow new deviations to update the envelope provide_series: [anomaly_score, y, yhat, yhat_lower, yhat_upper] queries: [cpu_seconds_total] schedulers: [periodic_online_weekly] # fit on two weekly cycles, then update online every 15m anomaly_score_outside_data_range: 1.5 # override default anomaly score outside expected data range clip_predictions: True # clip predictions to expected data range, i.e. [0, inf] for this query cpu_seconds_total seasonalities: [hod_smooth, dow_smooth] # where to read data from reader: class: vm datasource_url: https://play.victoriametrics.com/ tenant_id: 0:0 sampling_period: 30s # what data resolution to fetch from VictoriaMetrics /query_range endpoint workers: 0 # automatically choose bounded datasource concurrency latency_offset: 1ms query_from_last_seen_timestamp: False tz: UTC # timezone to use for queries without explicit timezone offset: 0s # offset to apply to all queries, e.g. to account for data delays, can be overridden on per-query basis queries: # aliases to MetricsQL expressions cpu_seconds_total: expr: avg(rate(node_cpu_seconds_total[5m])) by (mode) # step: 30s # if not set, will be equal to reader-level sampling_period data_range: [0, inf] # query-level business policy from v1.30.2 detection_direction: above_expected # query-level from v1.30.2; detect spikes only min_dev_from_expected: [0.01, 0.01] # query-level from v1.30.2 host_network_receive_errors: expr: rate(node_network_receive_errs_total[3m]) / rate(node_network_receive_packets_total[3m]) step: 15m # here we override per-query sampling_period to request way less data from VM TSDB data_range: [0, inf] # query-level business policy from v1.30.2 detection_direction: above_expected # query-level from v1.30.2; detect spikes only min_dev_from_expected: 0.0 # query-level from v1.30.2; absolute-deviation filtering is disabled # where to write data to writer: datasource_url: http://victoriametrics:8428/ tenant_id: 0:0 # for VictoriaMetrics cluster, can support multitenant metric_format: __name__: $VAR for: $QUERY_KEY # enable self-monitoring in pull and/or push mode monitoring: # pull: # Enable /metrics endpoint. # addr: 0.0.0.0 # port: 8490 push: # Enable pushing self-monitoring metrics url: http://victoriametrics:8428 push_frequency: 15m # how often to push self-monitoring metrics # configure vmanomaly server and UI settings server: port: 8490 path_prefix: /vmanomaly # optional path prefix for all HTTP routes max_concurrent_tasks: 4 # maximum number of concurrent anomaly detection tasks processed by backend use_reader_connection_settings: True # if True, use readers datasource_url and credentials for UI requests to datasource uvicorn_config: # optional Uvicorn server configuration log_level: warning该配置的核心语义如下schedulers定义两个调度器periodic_online每 30 秒推理一次、以 3 天窗口做一次性引导拟合periodic_online_weekly每 15 分钟推理一次、以 14 天窗口拟合覆盖两个周周期。models定义两个模型zscore只跑在host_network_receive_errors查询上、由periodic_online驱动envelope_weeklyTemporal Envelope 在线模型只跑在cpu_seconds_total查询上、由periodic_online_weekly驱动并额外配置了anomaly_score_outside_data_range: 1.5覆盖全局默认值。reader定义两个 MetricsQL 查询host_network_receive_errors通过step: 15m覆盖了 reader 级sampling_period显著降低从 TSDB 读取的数据量。writer通过metric_format控制输出命名__name__: $VAR使输出指标名为anomaly_score、y、yhat等for: $QUERY_KEY添加查询别名标签。monitoring开启 push 自监控每 15 分钟推送一次指标。server开启 vmanomaly 自带 UI/API端口 8490路径前缀/vmanomaly。说明本示例以及 settings.md 中的示例使用fit_every: 1000d作为仅引导一次bootstrap-only的调度。这适用于自带遗忘/反应机制的在线模型例如zscore_online配合decay 1。如果需要显式丢弃过期历史则应改用有限拟合周期——每次 fit 都会用配置的fit_window重置在线模型状态。深入各区块Reader 与 per-query 参数Reader 是数据的入口。class: vmVmReader通过 MetricsQL 从 VictoriaMetrics/Prometheus 读取class: vlogsVLogsReaderv1.26.0 起通过 LogsQL 的statspipe 从 VictoriaLogs/VictoriaTraces 读取。详见 reader.md。自 v1.13.0 起queries支持**按查询per-query**配置子字段并覆盖 reader 级参数这正是上例中step: 15m生效的原理。常用子字段包括exprMetricsQL/PromQL 表达式即/query_range?query%s接受的内容step该查询返回数据点的频率覆盖 reader 级sampling_perioddata_rangev1.15.1合法数据范围。数据落在范围外会得到高异常分数1默认1.01可用模型级anomaly_score_outside_data_range调整模型预测落在范围外则异常分数为 0detection_directionv1.30.2both/above_expected/below_expected控制只检测向上还是向下的偏差min_dev_from_expectedv1.30.2绝对偏差阈值|y - yhat|小于该值时异常分数置 0可配置标量或双元素列表下/上两个方向min_rel_dev_from_expectedv1.30.2相对偏差阈值|y - yhat| / |yhat|小于该值时异常分数置 0max_points_per_queryv1.17.0拆分长fit_window查询的子区间上限避免单个查询超时tzv1.18.0、tenant_idv1.19.0、offsetv1.25.3分别覆盖时区、租户与查询时间偏移。reader 级还有workersv1.30.20表示按查询数与 CPU 自动选择有界并发、fetch_timeout/processing_timeoutv1.30.0分别控制数据源请求与结果后处理超时、series_processing_batch_sizev1.29.7高基数查询建议 4–16等参数。深入各区块Scheduler 的三种工作模式调度器决定多久跑一次、跑哪个时间范围的数据。class有三种取值periodicPeriodicScheduler生产环境常用。周期性地对新数据推理并按fit_every周期性重训模型以对抗数据漂移。核心参数为fit_window训练时间范围至少 1 秒、infer_every推理频率至少 1 秒、fit_every重训频率缺省等于infer_every、start_from/tzv1.18.5指定首次 fit 的启动时间与时区配合restore_state: true可避免重启后长时间空转、scatter_infer_jobsv1.29.7把推理任务均匀分散到推理间隔内降低突发负载。oneoffOneoffScheduler运行一次即退出适合测试或对历史数据一次性回填。通过fit_start_iso/fit_end_iso或fit_start_s/fit_end_s与infer_start_iso/infer_end_iso或infer_start_s/infer_end_s显式指定拟合与推理时间窗。backtestingBacktestingScheduler模拟周期性调度但在历史数据上只跑一次后退出用于评估模型在过去的实际表现。v1.22.1 推荐inference_only: true模式——由from/to定义仅用于推理的时间窗训练窗自动取每个推理段之前的fit_windowv1.28.0 的exact: true使在线模型按infer_every的小批量时序精确回放生产行为。自 v1.11.0 起配置区块需命名为schedulers复数旧的扁平scheduler写法会被隐式转换为默认别名default_scheduler并保留向后兼容。深入各区块Model 的类型、公共参数与输出模型是异常检测的核心。模型沿两个维度分类按输入处理方式**单变量univariate**模型对每条时间序列各训练一个实例**多变量multivariate**模型对一组对齐的时间序列共享一个实例可捕获跨序列的集体异常。按更新策略**离线offline**模型仅在fit时全量重训**在线online**模型v1.15.0在每个infer_every步长上做增量更新即使只有一个数据点也能更新参数显著降低数据源读取压力。内置模型包括auto自动调参、temporal_envelope复杂运营数据的首选在线模型支持趋势/日历/节假日/预测、mad_online基于 t-digest 的稳健中位数绝对偏差、quantile_online在线季节性分位数、zscore_online在线 Z 分数、rolling_quantile、prophet、isolation_forest_multivariate、holtwinters、std等。其中 Prophet、Isolation Forest、Holt-Winters 已标记为计划弃用文档建议新部署迁移到对应的 Temporal Envelope 形式。所有模型共享的公共参数包括queriesv1.10.0选择该模型使用的 reader 查询不写则默认使用 reader 中全部查询。schedulersv1.11.0选择驱动该模型的调度器不写则默认挂到全部调度器。provide_seriesv1.12.0限制回写的输出列如[anomaly_score]timestamp列会被隐式加入。scalev1.20.0 支持双向以[scale_lower, scale_upper]分别缩放下/上置信区间宽度。clip_predictionsv1.20.0把yhat系列裁剪到data_range内。anomaly_score_outside_data_rangev1.20.0覆盖数据越界时的异常分数默认 1.01。decayv1.23.0仅在线模型指数遗忘因子取值(0.0, 1.0]1.0表示不衰减。groupbyv1.13.0仅多变量模型按标签值分组每组各训一个独立多变量模型。注意data_range、detection_direction、min_dev_from_expected、min_rel_dev_from_expected在模型级配置已自v1.30.2起弃用应迁移到reader.queries.alias下的查询级策略查询级显式值具有权威性模型级旧值仅在查询未定义时作为本地回退。vmanomaly的标准输出指标为anomaly_score主指标0–1 视为正常大于 1 判定异常且跨模型归一化、yhat预测期望值、yhat_lower/yhat_upper预测下/上界、y原始值。若infer收到 NaN 或无穷大输入对应anomaly_score为 NaN。深入各区块Writer 的指标格式化与多租户Writer 负责把模型输出写回 VictoriaMetrics其metric_format有两个必填键__name__必须包含$VAR占位符用于区分输出指标类型例如__name__: vmanomaly_$VAR会生成vmanomaly_anomaly_score、vmanomaly_yhat_lower等for通常填$QUERY_KEY为每条输出附加查询别名标签。其余键为用户自定义标签输入查询自带的标签如cpu1, deviceeth0, instancenode-exporter:9100会被原样继承到输出指标上。此外 writer 还支持 mTLSverify_tls/tls_cert_file/tls_key_file、BasicAuth、bearer token以及 v1.30.3 起的batch_max_series/batch_max_bytes/metric_prefix_cache_max_entries批量写入调优参数。多租户场景下tenant_id支持multitenant端点跨租户写入但需要注意聚合查询可能丢失vm_account_id路由标签而回落到默认租户0:0会打印警告。深入各区块Settings、Monitoring 与 ServerSettingssettings.md控制服务级行为n_workers与native_threads_per_workerv1.30.2控制进程级并行与数值库线程数restore_statev1.24.0使服务有状态重启后从$VMANOMALY_MODEL_DUMPS_DIR/vmanomaly.db恢复模型与调度器状态需开启磁盘模式且模型签名变化时自动重训retentionv1.28.1以ttlcheck_interval清理长期运行中累积的陈旧模型实例logger_levelsv1.25.3支持按组件前缀设置日志级别并支持热更新。Monitoringmonitoring.md提供 push 与 pull 两种自监控push可配置url、push_frequency默认 15m置空字符串可禁用定时推送仅保留 fit/infer 阶段推送、extra_labels等pull配置addr/port暴露/metrics。服务会产出vmanomaly_reader_*、vmanomaly_model_*、vmanomaly_writer_*、vmanomaly_config_reload*、vmanomaly_scheduler_*等系列自监控指标。Serverserver.md负责 REST API、/metrics端点与 Web UIport默认 8490、path_prefix可为所有路由加前缀如/vmanomaly后 UI 地址为http://localhost:8490/vmanomaly/vmui/、max_concurrent_tasks默认 2、ui_default_state可指定 UI 默认状态、use_reader_connection_settingsv1.29.2让 UI 复用 reader 的数据源连接凭据。v1.30.0 还提供了GET /api/v1/timeseries/characteristics序列特征分析与POST/GET/DELETE /api/v1/autotune/tasks异步共享调参任务端点。配置热重载Hot Reload自v1.25.0起vmanomaly支持无需重启进程地热重载配置文件。启用方式是在命令行加--watch参数详见 QuickStart.md 的 Command-line arguments 一节usage: vmanomaly.py [--license STRING | --licenseFile PATH] [--license.forceOffline] [--loggerLevel {DEBUG,INFO,WARNING,ERROR,FATAL}] [--watch] [-configCheckInterval DURATION] [--dryRun] [--outputSpec PATH] config [config ...]热重载最好与有状态服务stateful service配合使用——通过restore_state保留模型与调度器状态重启后无需重新训练模型、重新初始化调度器、重新读取数据最大化复用已有成果。自监控指标vmanomaly_config_reload_enabled在启用热重载时为1否则为0。[!WARNING] 自v1.29.5起基于文件系统事件的旧式热重载已被弃用改为基于内容轮询的方式原因是 Kubernetes ConfigMap 符号链接轮换等场景下事件投递不可靠。如果此前使用的是文件系统事件式热重载请改用--watch标志并按要求配置-configCheckInterval。热重载的工作原理服务按-configCheckInterval默认30s轮询被监听的.yml/.yaml文件内容v1.29.5。当检测到内容变化时会先等待防抖窗口然后重建全局配置并重新初始化各组件。vmanomaly_config_reloads_total指标会以statussuccess或statusfailure递增校验失败也会记录到日志。关键稳定性保证是如果重载失败服务会记录失败原因日志并继续沿用上一次有效的配置运行直到某次重载成功。这意味着新配置即使有错误服务也不会中断而是保持最后一份有效配置继续工作。在分片sharded部署中每次全局配置变更都会重新计算当前分片的归属。自 v1.30.4 起没有可运行任务的分片会保持存活但空闲后续重载若分配了任务它会在不重启进程的情况下恢复兼容的模型状态、创建调度器并开始执行。热重载使用的分片拓扑来自进程启动时的环境变量——变更分片数量、成员索引、副本因子或分配策略都需要编排层滚动升级或重启进程。分配策略方面ROUND_ROBIN在增删实体时可能移动规范有序子配置的后缀RENDEZVOUS在分片集合不变时保持无关分配的稳定性见 Scaling-vmanomaly.md 的分配策略指引。热重载示例假设服务以config.yaml启动内容如下settings: n_workers: 4 # number of workers to run models in parallel anomaly_score_outside_data_range: 5.0 # default anomaly score for anomalies outside expected data range restore_state: True # restore state from previous run, if available schedulers: periodic: class: periodic infer_every: 30s fit_every: 1000d # bootstrap-only schedule; use a finite cadence if accumulated state must be reset fit_window: 24h reader: datasource_url: https://play.victoriametrics.com/ tenant_id: 0:0 class: vm sampling_period: 30s queries: cpu_seconds_total: expr: avg(rate(node_cpu_seconds_total[5m])) by (mode) data_range: [0, inf] # step: 30s # if not set, will be equal to reader-level sampling_period host_network_receive_errors: expr: rate(node_network_receive_errs_total[3m]) / rate(node_network_receive_packets_total[3m]) step: 15s data_range: [0, inf] models: zscore: class: zscore_online z_threshold: 3.5 decay: 0.99 # gives more weight to recent data points, value should be in (0, 1], 1 means to give equal weight to all data provide_series: [anomaly_score] # if queries are not specified, all queries from reader will be used # if schedulers are not specified, all schedulers will be used writer: datasource_url: http://victoriametrics:8428/ tenant_id: 0:0 monitoring: push: url: http://victoriametrics:8428 push_frequency: 15m假设服务启动 15 分钟后reader.queries中cpu_seconds_total的查询表达式与频率发生了变化# ... (rest of the config remains unchanged) reader: # ... (rest of the reader config remains unchanged) queries: cpu_seconds_total: expr: avg(rate(node_cpu_seconds_total[10m])) by (mode) # changed lookback period data_range: [0, inf] step: 60s # changed step # ... (rest of the config remains unchanged)保存改动后热重载会自动检测config.yaml的内容变化并尝试重载。由于改动有效服务会记录成功日志并以statussuccess递增vmanomaly_config_reloads_total。重载后的实际效果是按需复用、只重训受影响的部分所有在host_network_receive_errors上训练的zscore_online模型实例仍然有效可继续直接对新数据点推理直到下一个fit_every触发所有在cpu_seconds_total上训练的zscore_online模型实例因查询表达式与频率变化而失效会以新的查询表达式与频率重新训练。环境变量占位符安全注入敏感配置自v1.25.0起配置文件中可以直接引用环境变量语法为标量字符串占位符%{ENV_NAME}。这对管理 API Key、数据库凭据等敏感信息特别有用——敏感值不必硬编码进配置文件而是由部署环境注入。例如设置环境变量VMANOMALY_URLhttp://localhost:8428后可在 reader 区块中写datasource_url: %{VMANOMALY_URL}启动时即被替换为实际值。注意如果引用的环境变量未设置或拼写有误占位符不会被替换可能导致配置校验失败或端点探测失败。因此建议在启动服务前确保所有必需的环境变量都已就绪。环境变量示例reader: class: vm datasource_url: %{VMANOMALY_URL} # will be replaced with the value of VMANOMALY_URL environment variable tenant_id: %{VMANOMALY_TENANT_ID} # will be replaced with the value of VMANOMALY_TENANT_ID environment variable bearer_token: %{VMANOMALY_BEARER_TOKEN} # will be replaced with the value of VMANOMALY_BEARER_TOKEN environment variable sampling_period: 30s writer: datasource_url: %{VMANOMALY_URL} # will be replaced with the value of VMANOMALY_URL environment variable tenant_id: %{VMANOMALY_TENANT_ID} # will be replaced with the value of VMANOMALY_TENANT_ID environment variable bearer_token: %{VMANOMALY_BEARER_TOKEN} # will be replaced with the value of VMANOMALY_BEARER_TOKEN environment variable # other config sections ...上例中同一组VMANOMALY_URL、VMANOMALY_TENANT_ID、VMANOMALY_BEARER_TOKEN被 reader 与 writer 同时引用既避免了重复书写也保证了读写两端凭据一致。小结与延伸阅读一份可用的vmanomaly配置可以归纳为四句话reader决定读什么、schedulers决定何时跑、models决定怎么判、writer决定写哪去再按需叠加settings并行/状态/保留、monitoring自监控与serverUI/API。在此基础上--watch-configCheckInterval提供滚动更新能力%{ENV_NAME}占位符保证敏感信息可安全注入而--dryRun可在不启动服务、不需要 license 的前提下提前校验整份配置含多 YAML 合并与 schema 校验是上线前必做的检查步骤。若想继续深入建议依次阅读仓库中的components/reader.md——VmReader 与 VLogsReader 的全部参数、per-query 参数与 MetricsQL/LogsQL 查询示例components/models.md——内置模型矩阵、公共参数、模型输出与自定义模型指南components/scheduler.md——periodic / oneoff / backtesting 三种调度器的完整参数components/settings.md——并行化、状态恢复与保留策略components/monitoring.md 与 components/server.md——自监控指标与 REST API/UIQuickStart.md——命令行参数、Docker 部署与 license 配置。【免费下载链接】VictoriaMetricsVictoriaMetrics: fast, cost-effective monitoring solution and time series database项目地址: https://gitcode.com/GitHub_Trending/vi/VictoriaMetrics创作声明：本文部分内容由AI辅助生成（AIGC），仅供参考