Files
junhong_cmp_fiber/migrations/000274_add_card_traffic_collection_unique.up.sql
邹进 1214181e78
All checks were successful
构建并部署到测试环境(无 SSH) / build-and-deploy (push) Successful in 16m2s
feat: 轮询观测入库异步化,调网关与入库分离,采集记录经消息攒批写入
- 轮询步骤异步模式下只调网关并发布 card.ingest 入库消息,入库消费者固定并发 30、按采集 ID 去重、可重试错误不限次数退避重投
- 采集记录经 card.collection 攒批写入,失败整批重投,新增 (observation_id, observed_at) 唯一索引
- 网络、实名观测增加乱序保护;轮询读查询走缓存,套餐状态查询各自计时且失败不缓存
- 入库积压告警与反压;系统配置 polling.ingest.mode 默认 sync 可回退;测试环境 nsqd 消息全部落盘

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-06 16:33:29 +09:00

9 lines
552 B
SQL
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
-- 采集记录经消息批量写入(add-polling-async-ingest):重投时同一采集只保留一条。
-- 分区表的唯一索引必须包含分区键 observed_at;父表上建索引会自动作用到各分区,建索引期间锁写入,请在低峰执行。
CREATE UNIQUE INDEX IF NOT EXISTS uk_card_traffic_collection_observation
ON tb_card_traffic_collection (observation_id, observed_at);
CREATE UNIQUE INDEX IF NOT EXISTS uk_card_traffic_failure_observation
ON tb_card_traffic_collection_failure (observation_id, observed_at);