feat: 轮询系统重构(分片队列 + 停复机统一 + Handler 拆分)
All checks were successful
构建并部署到测试环境(无 SSH) / build-and-deploy (push) Successful in 9m46s

【核心变更】

1. 停复机逻辑统一(StopResumeService)
   - 新增 EvaluateAndAct 统一入口,封装三条件停复机判断
   - 停机条件:无套餐(no_package) / 流量耗尽(traffic_exhausted) / 未实名(not_realname)
   - 复机条件:stop_reason 合规 + 有套餐且未耗尽 + 已实名或行业卡
   - 修复设备套餐 Bug:hasValidPackage 按 device_id 查套餐,而非仅 iot_card_id
   - 设备维度停复机加幂等锁(Redis SetNX,TTL 30s),防止多卡并发重复调 Gateway

2. Redis 分片队列(PollingQueueManager)
   - 新建 queue_manager.go,封装所有轮询 Redis 操作
   - 16 分片 Sorted Set,Key 格式:polling:shard:{shardID}:queue:{taskType}
   - Lua 脚本原子出队(ZRANGEBYSCORE + 分批 ZREM),消除竞态窗口
   - 新增背压检测:队列深度超 50 万时 Scheduler 跳过该分片
   - RemoveFromAllQueues 覆盖 4 种任务类型(含 protect)

3. Handler 拆分(polling_handler.go 1360行 → 5个专注文件)
   - polling_base.go:共享基类(并发控制/卡缓存/重入队)
   - polling_realname_handler.go:实名采集,实名 0→1 时立即触发复机
   - polling_carddata_handler.go:流量采集,保留跨月边界检测逻辑
   - polling_package_handler.go:套餐采集,委托 EvaluateAndAct 决策
   - polling_protect_handler.go:保护期一致性检查,保护期内强制修正

4. 配置管理(PollingConfigManager)
   - 新建 config_manager.go,从 scheduler.go 提取配置职责
   - 内存缓存 + 5 分钟定时刷新,刷新失败保留原缓存
   - 修复 getCardCondition:停机卡返回 suspended,不再错配 activated 配置

5. 渐进式初始化(CardInitializer)
   - 新建 initializer.go,分批加载(每批 10 万),批次间 sleep 500ms
   - 过滤 enable_polling=false 的卡,初始化完成前 Scheduler 不出队

6. 卡生命周期服务(PollingLifecycleService)
   - 新建 lifecycle_service.go,替代已删除的 callbacks.go 和 api_callback.go
   - OnCardCreated/OnCardEnabled/OnCardStatusChanged 入队前检查 enable_polling

7. Scheduler 精简(1000+行 → 227行)
   - 保留纯调度循环:scheduleLoop + processShardSchedule + enqueueBatch
   - 保留每 10 秒触发套餐过期检测和流量重置
   - 移除所有 DB 操作、配置加载、卡初始化逻辑

8. 轮询管控 API(enable_polling)
   - 新增 PUT /api/admin/assets/:id/polling-status 接口
   - 支持对设备/卡维度开关轮询,关闭后从所有分片队列移除

9. 数据库迁移
   - 000103:tb_device 新增 enable_polling 字段(boolean, NOT NULL, DEFAULT true)
   - 000104:新增 suspended 轮询配置,为 activated 配置补全 protect_check_interval

【文件统计】
- 新增:19 个文件(handler × 5、polling 组件 × 4、迁移 × 3 等)
- 修改:20 个文件(bootstrap 注入、store 接口、monitoring 适配分片等)
- 删除:3 个文件(polling_handler.go、callbacks.go、api_callback.go)

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
This commit is contained in:
2026-04-07 12:27:04 +08:00
parent 10fcc0b3c9
commit 434a8b0349
62 changed files with 7496 additions and 3023 deletions

View File

@@ -0,0 +1,172 @@
package task
import (
"context"
"time"
"github.com/bytedance/sonic"
"github.com/hibiken/asynq"
"go.uber.org/zap"
"github.com/break/junhong_cmp_fiber/internal/gateway"
iot_card_svc "github.com/break/junhong_cmp_fiber/internal/service/iot_card"
"github.com/break/junhong_cmp_fiber/internal/store/postgres"
"github.com/break/junhong_cmp_fiber/pkg/constants"
)
// PollingRealnameHandler 实名检查任务处理器
// 职责:调 Gateway 查实名状态 → 写 DB → 触发首次实名激活任务
// 不做:停复机决策(实名状态 0→1 时通过 EvaluateAndAct 触发 not_realname 复机)
type PollingRealnameHandler struct {
base *PollingBase
gateway *gateway.Client
iotCardStore *postgres.IotCardStore
asynqClient *asynq.Client
stopResumeSvc iot_card_svc.StopResumeServiceInterface
}
// NewPollingRealnameHandler 创建实名检查任务处理器
func NewPollingRealnameHandler(
base *PollingBase,
gw *gateway.Client,
iotCardStore *postgres.IotCardStore,
asynqClient *asynq.Client,
stopResumeSvc iot_card_svc.StopResumeServiceInterface,
) *PollingRealnameHandler {
return &PollingRealnameHandler{
base: base,
gateway: gw,
iotCardStore: iotCardStore,
asynqClient: asynqClient,
stopResumeSvc: stopResumeSvc,
}
}
// Handle 处理实名检查任务
func (h *PollingRealnameHandler) Handle(ctx context.Context, t *asynq.Task) error {
startTime := time.Now()
cardID, ok := parseTaskPayload(t.Payload(), h.base.logger)
if !ok {
return nil
}
if !h.base.acquireConcurrency(ctx, constants.TaskTypePollingRealname) {
h.base.logger.Debug("并发已满,重新入队", zap.Uint("card_id", cardID))
return h.base.requeueCard(ctx, cardID, constants.TaskTypePollingRealname)
}
defer h.base.releaseConcurrency(ctx, constants.TaskTypePollingRealname)
card, err := h.base.getCardWithCache(ctx, cardID)
if err != nil {
if isNotFound(err) {
return nil
}
h.base.logger.Error("获取卡信息失败", zap.Uint("card_id", cardID), zap.Error(err))
h.base.updateStats(ctx, constants.TaskTypePollingRealname, false, time.Since(startTime))
return h.base.requeueCard(ctx, cardID, constants.TaskTypePollingRealname)
}
if card.CardCategory == constants.CardCategoryIndustry {
return h.base.requeueCard(ctx, cardID, constants.TaskTypePollingRealname)
}
var newStatus int
if h.gateway != nil {
result, gwErr := h.gateway.QueryRealnameStatus(ctx, &gateway.CardStatusReq{CardNo: card.ICCID})
if gwErr != nil {
h.base.logger.Warn("查询实名状态失败",
zap.Uint("card_id", cardID), zap.String("iccid", card.ICCID), zap.Error(gwErr))
h.base.updateStats(ctx, constants.TaskTypePollingRealname, false, time.Since(startTime))
return h.base.requeueCard(ctx, cardID, constants.TaskTypePollingRealname)
}
if result.RealStatus {
newStatus = constants.RealNameStatusVerified
} else {
newStatus = constants.RealNameStatusNotVerified
}
} else {
newStatus = card.RealNameStatus
}
statusChanged := newStatus != card.RealNameStatus
isFirstRealname := statusChanged &&
card.RealNameStatus != constants.RealNameStatusVerified &&
newStatus == constants.RealNameStatusVerified
now := time.Now()
fields := map[string]any{"last_real_name_check_at": now}
if statusChanged {
fields["real_name_status"] = newStatus
}
if isFirstRealname {
fields["first_realname_at"] = now
}
if err := h.iotCardStore.UpdateFields(ctx, cardID, fields); err != nil {
h.base.logger.Error("更新卡实名信息失败", zap.Uint("card_id", cardID), zap.Error(err))
}
if statusChanged {
h.base.updateCardCache(ctx, cardID, map[string]any{"real_name_status": newStatus})
h.base.logger.Info("实名状态已变化",
zap.Uint("card_id", cardID),
zap.Int("old_status", card.RealNameStatus),
zap.Int("new_status", newStatus))
if isFirstRealname {
// 0→1首次实名触发套餐激活并立即复机评估不等待下一个 carddata/package 周期)
h.triggerFirstRealnameActivation(ctx, cardID)
if h.stopResumeSvc != nil {
freshCard, loadErr := h.iotCardStore.GetByID(ctx, cardID)
if loadErr == nil {
if evalErr := h.stopResumeSvc.EvaluateAndAct(ctx, freshCard); evalErr != nil {
h.base.logger.Warn("实名完成后触发复机评估失败",
zap.Uint("card_id", cardID), zap.Error(evalErr))
}
}
}
} else if newStatus == constants.RealNameStatusNotVerified {
// 1→0实名逆转立即触发停机评估不等待下一个 carddata/package 周期,否则最长延迟 1 小时)
h.base.logger.Warn("检测到实名逆转,立即触发停机评估",
zap.Uint("card_id", cardID))
if h.stopResumeSvc != nil {
freshCard, loadErr := h.iotCardStore.GetByID(ctx, cardID)
if loadErr == nil {
if evalErr := h.stopResumeSvc.EvaluateAndAct(ctx, freshCard); evalErr != nil {
h.base.logger.Warn("实名逆转后触发停机评估失败",
zap.Uint("card_id", cardID), zap.Error(evalErr))
}
}
}
}
}
h.base.updateStats(ctx, constants.TaskTypePollingRealname, true, time.Since(startTime))
return h.base.requeueCard(ctx, cardID, constants.TaskTypePollingRealname)
}
// triggerFirstRealnameActivation 首次实名后触发套餐激活任务
func (h *PollingRealnameHandler) triggerFirstRealnameActivation(ctx context.Context, cardID uint) {
if h.asynqClient == nil {
return
}
payload := map[string]any{
"carrier_type": "iot_card",
"carrier_id": cardID,
"activation_type": "realname",
"timestamp": time.Now().Unix(),
}
payloadBytes, err := sonic.Marshal(payload)
if err != nil {
h.base.logger.Warn("序列化首次实名激活任务载荷失败", zap.Uint("card_id", cardID), zap.Error(err))
return
}
task := asynq.NewTask(constants.TaskTypePackageFirstActivation, payloadBytes,
asynq.MaxRetry(3),
asynq.Timeout(30*time.Second),
asynq.Queue(constants.QueueDefault),
)
if _, err := h.asynqClient.EnqueueContext(ctx, task); err != nil {
h.base.logger.Warn("提交首次实名激活任务失败", zap.Uint("card_id", cardID), zap.Error(err))
}
}