feat: 轮询系统重构(分片队列 + 停复机统一 + Handler 拆分)
All checks were successful
构建并部署到测试环境(无 SSH) / build-and-deploy (push) Successful in 9m46s

【核心变更】

1. 停复机逻辑统一(StopResumeService)
   - 新增 EvaluateAndAct 统一入口,封装三条件停复机判断
   - 停机条件:无套餐(no_package) / 流量耗尽(traffic_exhausted) / 未实名(not_realname)
   - 复机条件:stop_reason 合规 + 有套餐且未耗尽 + 已实名或行业卡
   - 修复设备套餐 Bug:hasValidPackage 按 device_id 查套餐,而非仅 iot_card_id
   - 设备维度停复机加幂等锁(Redis SetNX,TTL 30s),防止多卡并发重复调 Gateway

2. Redis 分片队列(PollingQueueManager)
   - 新建 queue_manager.go,封装所有轮询 Redis 操作
   - 16 分片 Sorted Set,Key 格式:polling:shard:{shardID}:queue:{taskType}
   - Lua 脚本原子出队(ZRANGEBYSCORE + 分批 ZREM),消除竞态窗口
   - 新增背压检测:队列深度超 50 万时 Scheduler 跳过该分片
   - RemoveFromAllQueues 覆盖 4 种任务类型(含 protect)

3. Handler 拆分(polling_handler.go 1360行 → 5个专注文件)
   - polling_base.go:共享基类(并发控制/卡缓存/重入队)
   - polling_realname_handler.go:实名采集,实名 0→1 时立即触发复机
   - polling_carddata_handler.go:流量采集,保留跨月边界检测逻辑
   - polling_package_handler.go:套餐采集,委托 EvaluateAndAct 决策
   - polling_protect_handler.go:保护期一致性检查,保护期内强制修正

4. 配置管理(PollingConfigManager)
   - 新建 config_manager.go,从 scheduler.go 提取配置职责
   - 内存缓存 + 5 分钟定时刷新,刷新失败保留原缓存
   - 修复 getCardCondition:停机卡返回 suspended,不再错配 activated 配置

5. 渐进式初始化(CardInitializer)
   - 新建 initializer.go,分批加载(每批 10 万),批次间 sleep 500ms
   - 过滤 enable_polling=false 的卡,初始化完成前 Scheduler 不出队

6. 卡生命周期服务(PollingLifecycleService)
   - 新建 lifecycle_service.go,替代已删除的 callbacks.go 和 api_callback.go
   - OnCardCreated/OnCardEnabled/OnCardStatusChanged 入队前检查 enable_polling

7. Scheduler 精简(1000+行 → 227行)
   - 保留纯调度循环:scheduleLoop + processShardSchedule + enqueueBatch
   - 保留每 10 秒触发套餐过期检测和流量重置
   - 移除所有 DB 操作、配置加载、卡初始化逻辑

8. 轮询管控 API(enable_polling)
   - 新增 PUT /api/admin/assets/:id/polling-status 接口
   - 支持对设备/卡维度开关轮询,关闭后从所有分片队列移除

9. 数据库迁移
   - 000103:tb_device 新增 enable_polling 字段(boolean, NOT NULL, DEFAULT true)
   - 000104:新增 suspended 轮询配置,为 activated 配置补全 protect_check_interval

【文件统计】
- 新增:19 个文件(handler × 5、polling 组件 × 4、迁移 × 3 等)
- 修改:20 个文件(bootstrap 注入、store 接口、monitoring 适配分片等)
- 删除:3 个文件(polling_handler.go、callbacks.go、api_callback.go)

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
This commit is contained in:
2026-04-07 12:27:04 +08:00
parent 10fcc0b3c9
commit 434a8b0349
62 changed files with 7496 additions and 3023 deletions

View File

@@ -81,3 +81,17 @@ func (s *CarrierStore) List(ctx context.Context, opts *store.QueryOptions, filte
return carriers, total, nil
}
// GetDataResetDay 获取运营商上游流量重置日1-28查询失败默认返回 1
func (s *CarrierStore) GetDataResetDay(ctx context.Context, carrierID uint) int {
var day int
err := s.db.WithContext(ctx).
Model(&model.Carrier{}).
Select("data_reset_day").
Where("id = ?", carrierID).
Scan(&day).Error
if err != nil || day == 0 {
return 1
}
return day
}

View File

@@ -243,3 +243,10 @@ func (s *DeviceStore) UpdateRechargeTrackingFields(ctx context.Context, deviceID
"first_recharge_triggered_by_series": triggeredJSON,
}).Error
}
// UpdatePollingStatus 更新设备的轮询启用状态
func (s *DeviceStore) UpdatePollingStatus(ctx context.Context, deviceID uint, enablePolling bool) error {
return s.db.WithContext(ctx).Model(&model.Device{}).
Where("id = ?", deviceID).
Update("enable_polling", enablePolling).Error
}

View File

@@ -110,6 +110,20 @@ func (s *IotCardStore) Update(ctx context.Context, card *model.IotCard) error {
return s.db.WithContext(ctx).Save(card).Error
}
// UpdateFields 按 ID 部分更新卡字段,供轮询系统的 Handler 使用
func (s *IotCardStore) UpdateFields(ctx context.Context, cardID uint, fields map[string]any) error {
return s.db.WithContext(ctx).Model(&model.IotCard{}).
Where("id = ?", cardID).
Updates(fields).Error
}
// UpdatePollingStatus 更新卡的轮询启用状态
func (s *IotCardStore) UpdatePollingStatus(ctx context.Context, cardID uint, enablePolling bool) error {
return s.db.WithContext(ctx).Model(&model.IotCard{}).
Where("id = ?", cardID).
Update("enable_polling", enablePolling).Error
}
func (s *IotCardStore) Delete(ctx context.Context, id uint) error {
return s.db.WithContext(ctx).Delete(&model.IotCard{}, id).Error
}
@@ -927,6 +941,65 @@ func (s *IotCardStore) ExistsByVirtualNoBatch(ctx context.Context, virtualNos []
return result, nil
}
// ==================== 轮询系统专用查询 ====================
// ListByDeviceIDAndNetworkStatus 查询设备下指定网络状态的所有卡
// 通过 tb_device_sim_binding 关联查询(项目禁止外键约束,用子查询实现)
func (s *IotCardStore) ListByDeviceIDAndNetworkStatus(ctx context.Context, deviceID uint, networkStatus int) ([]*model.IotCard, error) {
var cards []*model.IotCard
err := s.db.WithContext(ctx).
Where("id IN (SELECT iot_card_id FROM tb_device_sim_binding WHERE device_id = ? AND bind_status = 1 AND deleted_at IS NULL)", deviceID).
Where("network_status = ? AND deleted_at IS NULL", networkStatus).
Find(&cards).Error
return cards, err
}
// ListByDeviceIDAndPollingStopReasons 查询设备下因轮询原因停机的所有卡
// 用于 resumeDeviceCards只复机由轮询系统停机的卡不干预手动停机
func (s *IotCardStore) ListByDeviceIDAndPollingStopReasons(ctx context.Context, deviceID uint) ([]*model.IotCard, error) {
pollingReasons := []string{
constants.StopReasonTrafficExhausted,
constants.StopReasonNoPackage,
constants.StopReasonNotRealname,
}
var cards []*model.IotCard
err := s.db.WithContext(ctx).
Where("id IN (SELECT iot_card_id FROM tb_device_sim_binding WHERE device_id = ? AND bind_status = 1 AND deleted_at IS NULL)", deviceID).
Where("network_status = 0 AND stop_reason IN ? AND deleted_at IS NULL", pollingReasons).
Find(&cards).Error
return cards, err
}
// UpdateStopReason 更新卡的停机原因字段
func (s *IotCardStore) UpdateStopReason(ctx context.Context, cardID uint, reason string) error {
return s.db.WithContext(ctx).
Model(&model.IotCard{}).
Where("id = ?", cardID).
Update("stop_reason", reason).Error
}
// CountForPolling 统计启用轮询的卡总数(仅供轮询初始化使用,不应用数据权限过滤)
func (s *IotCardStore) CountForPolling(ctx context.Context) (int64, error) {
var total int64
err := s.db.WithContext(ctx).
Model(&model.IotCard{}).
Where("enable_polling = true AND deleted_at IS NULL").
Count(&total).Error
return total, err
}
// ListForPollingBatch 分批查询启用轮询的卡,按 id ASC 游标翻页(仅供轮询初始化使用,不应用数据权限过滤)
// lastID: 上批最后一条卡的 id首次传 0limit: 每批大小
func (s *IotCardStore) ListForPollingBatch(ctx context.Context, lastID uint, limit int) ([]*model.IotCard, error) {
var cards []*model.IotCard
err := s.db.WithContext(ctx).
Where("id > ? AND enable_polling = true AND deleted_at IS NULL", lastID).
Order("id ASC").
Limit(limit).
Find(&cards).Error
return cards, err
}
// ==================== 列表计数缓存 ====================
func (s *IotCardStore) getCachedCount(ctx context.Context, table string, filters map[string]any) (int64, bool) {

View File

@@ -172,6 +172,51 @@ func (s *PackageUsageStore) ResetDataUsage(ctx context.Context, id uint, lastRes
}).Error
}
// HasValidByCarrier 检查载体是否有有效套餐status IN 待生效/生效中)
// 修复Bug1通过 carrierType 动态切换 device_id / iot_card_id 查询
// carrierType: "iot_card" 或 "device"
func (s *PackageUsageStore) HasValidByCarrier(ctx context.Context, carrierType string, carrierID uint) (bool, error) {
var count int64
validStatuses := []int{
constants.PackageUsageStatusPending, // 待生效
constants.PackageUsageStatusActive, // 生效中
}
query := s.db.WithContext(ctx).Model(&model.PackageUsage{}).
Where("status IN ?", validStatuses)
if carrierType == "device" {
query = query.Where("device_id = ?", carrierID)
} else {
query = query.Where("iot_card_id = ?", carrierID)
}
if err := query.Count(&count).Error; err != nil {
return false, err
}
return count > 0, nil
}
// GetActiveOrDepletedByCarrier 查询载体的活跃或耗尽套餐(用于流量耗尽检测)
// ORDER BY status ASC 优先返回活跃套餐status=1有活跃套餐时先评估活跃套餐的流量上限
// 返回 gorm.ErrRecordNotFound 表示无活跃或耗尽套餐(不判定为流量耗尽)
func (s *PackageUsageStore) GetActiveOrDepletedByCarrier(ctx context.Context, carrierType string, carrierID uint) (*model.PackageUsage, error) {
var pkg model.PackageUsage
query := s.db.WithContext(ctx).Model(&model.PackageUsage{}).
Where("status IN ?", []int{
constants.PackageUsageStatusActive, // 生效中
constants.PackageUsageStatusDepleted, // 已用完
}).
Order("status ASC").
Limit(1)
if carrierType == "device" {
query = query.Where("device_id = ?", carrierID)
} else {
query = query.Where("iot_card_id = ?", carrierID)
}
if err := query.First(&pkg).Error; err != nil {
return nil, err
}
return &pkg, nil
}
// ListByCarrier 查询载体的所有套餐使用记录(包括所有状态)
func (s *PackageUsageStore) ListByCarrier(ctx context.Context, carrierType string, carrierID uint) ([]*model.PackageUsage, error) {
var usages []*model.PackageUsage