26 Commits

Author SHA1 Message Date
邹进
9183863fab docs: 按运营商账户调度任务进度
All checks were successful
构建并部署到测试环境(无 SSH) / build-and-deploy (push) Successful in 17m39s
2026-10-25 18:57:32 +09:00
邹进
c741f1751d feat: 按运营商账户调度的后台接口与重算到期
- /polling-carrier-params 列表(含默认标记与消费能力)、详情、保存(执行上限不得大于订阅进程 max_in_flight 之和)、删除;/polling-check-scenes 列表与修改(检查类型、递增的确认延迟、冷却);/polling-failure-alerts 列表、按账户与错误类别汇总、详情、标记已处理;GET /iot-cards/:id/polling-state 单卡轮询状态;GET /polling-stats/carrier-queues 按账户队列与执行监控
- 读取仅超级管理员与平台账号,修改仅超级管理员(告警处理平台账号可用),均写统一审计;新增审计动作与资源
- carrier 模式下重算到期时间写入卡到期信息与账户队列,只提前不推后
2026-10-25 18:56:32 +09:00
邹进
0d966249b8 feat: 业务事件与读入口在按运营商账户调度时改为请求检查
- 请求检查器按场景配置(tb_polling_check_scene,缓存 30 秒)把卡的检查类型设为立即到期放入账户优先队列,同卡合并、冷却期内不请求、按场景追加延迟确认;设备按有效绑定卡展开
- 业务事件仍在原事务中写 Outbox(提交安全),carrier 模式下 Outbox 消费者把订单支付、套餐激活、自动续费购包、停复机成功事件转为请求检查,不再起三连查观测序列
- 读入口(C 端资产详情、后台资产实时状态、实名链接、设备操作)经模式接缝改为请求检查;设备信息同步保持原观测;OpenAPI 查询仍不触发
- legacy 模式下全部沿用原观测序列
2026-10-25 18:49:36 +09:00
邹进
2fe7c1f8ac feat: 按运营商账户调度的分发器、卡任务执行器与模式切换
- 四个轮询处理器拆为步骤函数 Step(返回结果分类),旧 Asynq Handle 只做并发控制与原有重入队映射,行为不变;carrier 模式下旧 handler 直接结束
- 网关限流增加等待模式(ctx 标记),NSQ 消费者等名额到单步超时,超时按限流处理不计失败;记录限流等待次数
- 卡任务执行器:核对 token 并抢执行权,按配置步骤顺序串行执行到期检查(单步超时、总超时、panic 恢复、流量已触发评估时跳过套餐),执行中续期并 Touch,续期失败立即停止;收尾写回到期、连续失败、退避、延迟确认与告警
- 分发器:每秒按账户抢分发锁领取(执行上限按消费能力收紧、优先占比、按限流组剩余 QPS 估算),发布到 polling.card.{账户},发布失败释放;消费旧优先提示与手动触发;心跳与处理中标记到期计数
- NSQ 生产者与按账户动态订阅的消费者管理器,优雅退出宽限期
- 失败告警异步写 tb_polling_failure_alert(同卡同类型一条未处理,成功转已恢复),账户窗口内告警数超阈值产生账户级告警
- 模式监听:切 carrier 时旧分片调度停止出队,加锁初始化新结构后开始领取并订阅;回 legacy 时停止消费、重建旧分片队列;生命周期(API 与 Worker)在 carrier 模式写账户队列
2026-10-25 18:44:23 +09:00
邹进
e7f8c4bae9 docs: 统一停复机规则转正测试环境验证通过
Some checks are pending
构建并部署到测试环境(无 SSH) / build-and-deploy (push) Waiting to run
2026-10-25 18:29:56 +09:00
邹进
dae44bed30 feat: 按运营商账户调度的迁移、配置与账户队列 Lua
- 迁移 000251~000254:运营商账户调度参数、请求检查场景(预置 8 个)、卡轮询失败告警表,以及调度模式、步骤、退避、告警等受控系统配置(默认 legacy,行为不变);并发配置表注释标记废弃
- 受控系统配置注册与 JSON 结构校验:步骤只能是 4 种检查类型且不重复、退避序列递增
- yaml 增加 NSQ 地址、msg_timeout、租约、续期间隔、分发间隔、默认执行上限与 max_in_flight;引入 go-nsq
- 测试环境 docker-compose 增加 nsqlookupd 与 nsqd,Worker 注入 NSQ 地址
- 账户队列 Redis 结构与 Lua:领取(执行上限、优先占比、按限流组剩余 QPS 估算)、抢执行权、续期、收尾、请求检查(合并、冷却、执行中打优先标记)、移出、发布失败释放
2026-10-25 18:28:42 +09:00
邹进
f49baff7c2 feat: 停复机新规则转正,删除总开关与旧口径
All checks were successful
构建并部署到测试环境(无 SSH) / build-and-deploy (push) Successful in 9m35s
- 删除总开关 stop_resume.unified_rule_enabled(配置、加载、部署工作流与 docker-compose.prod.yml 环境变量)
- 复机不看停机原因:删除只对轮询停因复机的旧分支与 ListByDeviceIDAndPollingStopReasons;设备复机取全部停机卡
- 实名门槛只按卡所属通道实名管控判断,删除行业卡与卡/设备实名策略放行;暂停轮询的卡不自动停复机无条件生效
- 通道阈值新周期解锁后的复机条件固定为新规则复机条件
- 资产自动续费复机同步新规则:不看停因、暂停轮询不复机、设备取全部停机卡、复机成功清除停因
- 实名轮询不再按行业卡跳过;所属运营商账户未开启实名管控时跳过查询,但卡未实名且所在资产有等待实名激活的套餐时照常查询,避免囤货套餐无法激活
2026-10-25 18:13:51 +09:00
邹进
457abaabd7 docs: 统一停复机规则第一次发布测试环境验证通过
All checks were successful
构建并部署到测试环境(无 SSH) / build-and-deploy (push) Successful in 1m35s
2026-10-25 18:06:14 +09:00
邹进
af8249158d feat: 停复机规则演练与终态自动关闭轮询
All checks were successful
构建并部署到测试环境(无 SSH) / build-and-deploy (push) Successful in 9m26s
统一停复机规则转正的第一次发布:总开关仍关闭,现有停复机行为不变。

- 新增停复机规则演练:GET /stop-resume-rule/dry-run 按新规则对开机与停机卡只判断不执行,返回运营商账户 × 停机原因 × 判断结果汇总与分页明细,可假设通道实名管控配置;POST /stop-resume-rule/dry-run/export 导出明细(场景 stop_resume_rule_dry_run);仅超级管理员与平台账号
- 风险停机、已销户关闭轮询不再限于独立卡,只关该卡自身轮询,不级联设备
- 错误分类规则增加 auto_disable_polling,仅终态类别可勾选;停复机(含设备停复机)命中该规则时不重试、关闭该卡轮询、回写网关扩展状态并写审计 iot_card.terminal_disable_polling
- 迁移 000250 加列与约束,并为"已销户""仅支持风险复机"置位
2026-10-25 17:52:40 +09:00
邹进
fad7dbfdff docs: 设备保护期下线测试环境验证通过
All checks were successful
构建并部署到测试环境(无 SSH) / build-and-deploy (push) Successful in 1m39s
2026-10-25 17:35:54 +09:00
邹进
79b7580554 feat: 下线设备保护期
All checks were successful
构建并部署到测试环境(无 SSH) / build-and-deploy (push) Successful in 9m30s
- 后台设备停复机不再设置 1 小时保护期,也不再因保护期拒绝相反操作;单卡手动停复机去掉保护期检查
- 删除保护期一致性轮询(polling:protect)及其强制停复机方法;调度、初始化、监控、匹配预览不再包含保护期;已入队的旧任务由空处理函数直接确认,下一次发布移除
- 轮询配置接口不再接受与返回 protect_check_interval;并发配置不再返回 protect,查询、更新、重置该类型时拒绝
- 资产详情 device_protect_status 固定返回 none 并标记废弃,兼容老版本客户端
- 删除保护期 Redis key 与时长常量;停机原因 protect_period 保留供历史数据与开放接口映射
- 迁移 000249 只把 protect_check_interval、last_protect_check_at 列注释标记为已废弃,不删列不改数据
2026-10-25 17:19:10 +09:00
邹进
4de67f6dc3 fix: 设备管理接口不再拦截代理与企业账号
Some checks failed
构建并部署到测试环境(无 SSH) / build-and-deploy (push) Has been cancelled
设备类型选项接口用 router.Group("/devices", 拦截函数) 限制仅平台访问,Fiber 会把该函数注册为整个 /devices 前缀的中间件,导致代理、企业账号的设备列表、设备下的卡、分配回收等接口全部返回 403。改为在 ListDeviceTypes 内部判断用户类型,其他设备接口恢复原有权限。
2026-10-25 17:11:10 +09:00
邹进
a855742752 fix: 关闭轮询的卡不再重新入队
All checks were successful
构建并部署到测试环境(无 SSH) / build-and-deploy (push) Successful in 9m32s
关闭轮询时会移出队列并清除卡缓存,但执行收尾重入队、卡状态检查补齐套餐任务、限流重入队
都不判断轮询开关,执行中途被关闭的卡会被放回队列并互相补齐,关闭轮询无法生效。

- 卡缓存写入 enable_polling,旧缓存缺字段时从库补读并回写
- 重入队、补齐缺失任务、限流重入队前判断轮询开关,已关闭则不入队
2026-10-24 18:10:58 +09:00
邹进
2ddaffe373 docs: 轮询配置接口文档与实际行为对齐
All checks were successful
构建并部署到测试环境(无 SSH) / build-and-deploy (push) Successful in 9m57s
- 匹配预览 GET 只收 query iccid,POST 只收请求体,去掉 POST 文档中多余的 query 参数
- 预览草稿 overrides 改为独立结构,字段全部可选,最多 20 条
- clear_fields 用具名类型生成数组元素枚举;ordered_ids 说明须为正整数且不重复
2026-10-24 17:41:52 +09:00
邹进
8fc3485061 chore: 激活前流量结算记录开始与完成日志
All checks were successful
构建并部署到测试环境(无 SSH) / build-and-deploy (push) Successful in 9m46s
2026-10-24 17:31:32 +09:00
邹进
ec5c1996a7 fix: 重算到期时间不抵消失败退避,重置日前统计只计有生效套餐的卡
Some checks failed
构建并部署到测试环境(无 SSH) / build-and-deploy (push) Has been cancelled
- 上次检查时间只在查询成功时更新,按"上次检查 + 新间隔"会把失败或限流延后的卡拉到立即执行;
  过期或从未检查时候选改为"现在 + 新间隔",只提前排队超过一个新间隔的成员
- 重置日前加密统计只记录有生效套餐的卡,与"有生效套餐但窗口内未查到"口径一致
2026-10-24 17:28:18 +09:00
邹进
98017ac053 feat: 分层轮询、重置日前加密与激活前流量结算
All checks were successful
构建并部署到测试环境(无 SSH) / build-and-deploy (push) Successful in 9m37s
- 轮询配置新增套餐状态、资产状态、重置日前窗口与窗口内流量间隔(迁移 248),间隔三态:正数按间隔、0 不检查、空继承
- 配置按 priority, id 排序合并;在库与已销售无套餐未匹配时告警不入队
- 新增优先级整体重排、匹配预览、立即重算到期时间(异步只提前)、分层统计与重置日前加密统计接口
- 已切换日快照计量的卡流量按观测时间归属,激活前用量只记已结算
- 订单购买与实名激活补齐资产级激活锁,激活前对已切换卡结算一次流量,失败只告警不阻塞,结算不触发通道阈值判定
2026-10-24 17:06:38 +09:00
邹进
02a842f066 fix: 网关限流系统配置 Key 改为三段式,修复注册失败
All checks were successful
构建并部署到测试环境(无 SSH) / build-and-deploy (push) Successful in 9m35s
- 受控系统配置要求 Key 至少三段,原两段式 Key(如 polling.rate_limit_mode)注册被拒,后台不可见、限流器只能使用默认值
- Key 改为 polling.gateway.* 与 polling.stop_resume_record.*;新增迁移 000247 将已落库的旧 Key 改名并保留取值
- 后续提案中规划的开关同步改为三段式(polling.scheduler.mode、polling.stop_resume.queue_enabled)
2026-10-24 16:17:34 +09:00
邹进
9b0930db9f chore: 重新触发测试环境部署(磁盘清理后重跑)
All checks were successful
构建并部署到测试环境(无 SSH) / build-and-deploy (push) Successful in 11m59s
2026-10-24 15:38:40 +09:00
邹进
78220deb0e docs: 网关限流接口文档补充操作、类别与状态枚举
Some checks failed
构建并部署到测试环境(无 SSH) / build-and-deploy (push) Failing after 15m23s
- 限流组与错误分类的操作数组元素改为具名类型 GatewayOperation,接口文档按元素给出可选值
- 匹配方式、错误类别、状态字段补充 enum 标签;高优先级操作说明补充取值与必填条件
2026-10-24 15:22:17 +09:00
邹进
42e881576d feat: 卡网关按运营商账户限流并记录停复机调用
All checks were successful
构建并部署到测试环境(无 SSH) / build-and-deploy (push) Successful in 9m33s
- 新增网关限流:按本系统运营商账户(卡 carrier_id)→ 限流组 → 操作限制每秒请求数,每次 HTTP 尝试计量,默认限流、仅后台人工点击/通道阈值/自动续费/手动刷新豁免;支持只统计与拦截两种模式,默认只统计
- Asynq 遇限流不计失败、固定短延迟重试;轮询任务与停复机业务重试遇限流短延迟重排
- 停机、复机不再使用网关内部重试,只发一次请求
- 网关返回结构化错误;新增上游错误分类规则表与测试接口
- 新增按月分区的停复机记录表:系统自动停复机只写记录,不再写统一审计与集成日志;普通轮询查询不再写集成日志
- 新增限流组、错误分类、停复机记录、按账户调用统计与枚举后台接口
- 迁移 000243~000246;同步 OpenSpec 提案(含后续 6 个 change)与技术方案、工程约束
2026-10-24 15:06:41 +09:00
邹进
9e45097aa2 fix: 富友订单支持退款,0 元退款不强制客户收款材料映射
All checks were successful
构建并部署到测试环境(无 SSH) / build-and-deploy (push) Successful in 9m20s
- decideRefundMethods 将 fuiou 与 wechat、alipay 按线上支付处理,修复富友订单报"该订单的支付方式不支持退款"
- 客户收款信息退款仅在金额大于 0 时强制映射并提交客户收款信息与凭证;0 元退款允许客户收款信息为空

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 17:00:36 +09:00
邹进
437d1c2c2c feat: 支持 0 元退款(仅作废权益不退钱)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 11:18:24 +09:00
邹进
1f28ac3151 chore: 测试环境开启停复机新规则总开关
All checks were successful
构建并部署到测试环境(无 SSH) / build-and-deploy (push) Successful in 1m37s
部署工作流写入 JUNHONG_STOP_RESUME_UNIFIED_RULE_ENABLED=true,用于测试环境验证
复机不看停机原因、实名按通道实名管控判断、暂停轮询的卡不自动停复机。
2026-10-20 18:13:18 +09:00
邹进
6954ad6693 chore: 测试环境开启设备 10 的卡流量日快照计量灰度
All checks were successful
构建并部署到测试环境(无 SSH) / build-and-deploy (push) Successful in 10m0s
部署工作流写入 JUNHONG_TRAFFIC_CALC_MODE=gray 与 JUNHONG_TRAFFIC_CALC_CARDS=device:10;
docker-compose 为 API 与 Worker 透传计量与停复机新规则开关,未配置时保持原行为。
2026-10-20 17:58:18 +09:00
邹进
96771a204f feat: 卡流量改为日快照计量并统一停复机规则
Some checks failed
构建并部署到测试环境(无 SSH) / build-and-deploy (push) Has been cancelled
卡流量计量(按名单灰度,默认 off):
- 新增卡日快照与切换水位表(000241),按通道上游流量重置日换算自然月、重置日防旧值
- 已切换的卡按月对账产生增量,无生效中套餐只结算不发事件;第一次切换写水位
- 已切换卡读数有效即判定通道阈值,卡表本月用量用快照覆盖,日流量读快照
- 配置 JUNHONG_TRAFFIC_CALC_MODE/CARDS、JUNHONG_TRAFFIC_FLOW_SNAPSHOT_ENABLED

停复机规则(总开关 JUNHONG_STOP_RESUME_UNIFIED_RULE_ENABLED,默认关闭):
- 复机不看停机原因,持通道阈值锁仍拒绝;暂停轮询的卡不自动停复机
- 通道新增实名管控 realname_control_enabled(000242,默认 0),开启后不区分行业卡与实名策略
- 实名管控从 0 改 1 需确认,修改并入通道更新审计

OpenSpec Change: rebuild-card-traffic-metering-and-stop-resume
2026-10-20 17:49:26 +09:00
240 changed files with 17619 additions and 829 deletions

View File

@@ -60,6 +60,9 @@ jobs:
printf 'JUNHONG_APPROVAL_LEGACY_OFFLINE_RECHARGE_PAY_ENABLED=true\n'
printf 'JUNHONG_WORKER_ROLE=all\n'
printf 'JUNHONG_WORKER_INSTANCE_NAME=test-worker-all-1\n'
# 卡流量日快照计量灰度(rebuild-card-traffic-metering-and-stop-resume):仅测试设备 10 走新算法
printf 'JUNHONG_TRAFFIC_CALC_MODE=gray\n'
printf 'JUNHONG_TRAFFIC_CALC_CARDS=device:10\n'
} > ${{ env.DEPLOY_DIR }}/.env
# 强制更新 docker-compose.prod.yml(确保使用最新配置)

View File

@@ -31,12 +31,16 @@ func generateOpenAPIDocs(outputPath string, logger *zap.Logger) {
handlers.BusinessUserGroup = admin.NewBusinessUserGroupHandler(nil, nil)
handlers.ShopBusinessOwnerImport = admin.NewShopBusinessOwnerImportHandler(nil)
handlers.PhoneAssetAssociation = admin.NewPhoneAssetAssociationHandler(nil, nil)
// 网关限流 Handler 必须同时进入文档工厂,避免新增管理接口遗漏文档注册。
handlers.GatewayRateLimit = admin.NewGatewayRateLimitHandler(nil, nil, nil)
handlers.CarrierScheduler = admin.NewCarrierSchedulerHandler(nil, nil)
// 套餐真流量预警 Handler 必须同时进入文档工厂,避免新增管理接口遗漏文档注册。
handlers.PackageTrafficAlert = admin.NewPackageTrafficAlertHandler(nil, nil, nil, nil)
// 资产钱包自动续费配置 Handler 必须同时进入文档工厂,避免新增管理接口遗漏文档注册。
handlers.AssetAutoRenewal = admin.NewAssetAutoRenewalConfigHandler(nil, nil)
// 运营报表 Handler 必须同时进入文档工厂,避免新增管理接口遗漏文档注册。
handlers.OperationsReport = admin.NewOperationsReportHandler(nil, nil, nil)
handlers.StopResumeRule = admin.NewStopResumeRuleHandler(nil, nil, nil)
handlers.ClientPopup = apphandler.NewClientPopupHandler(nil, nil, nil)
handlers.H5PopupConfiguration = admin.NewH5PopupConfigurationHandler(nil, nil, nil)
// 企业微信 Handler 在此显式装配,避免新增管理接口遗漏文档注册。

View File

@@ -22,6 +22,7 @@ import (
"github.com/break/junhong_cmp_fiber/internal/bootstrap"
"github.com/break/junhong_cmp_fiber/internal/gateway"
"github.com/break/junhong_cmp_fiber/internal/infrastructure/gatewaylimit"
internalMiddleware "github.com/break/junhong_cmp_fiber/internal/middleware"
"github.com/break/junhong_cmp_fiber/internal/routes"
"github.com/break/junhong_cmp_fiber/internal/service/verification"
@@ -71,6 +72,11 @@ func main() {
// 9. 初始化 Gateway 客户端(可选)
gatewayClient := initGateway(cfg, appLogger)
// 卡网关调用按本系统运营商账户限流与统计(默认只统计不拦截)。
gatewayLimit := gatewaylimit.NewRuntime(db, redisClient, appLogger)
if gatewayClient != nil {
gatewayClient.SetCallGuard(gatewayLimit.Guard)
}
// 10. 初始化所有业务组件(通过 Bootstrap)
result, err := bootstrap.Bootstrap(&bootstrap.Dependencies{
@@ -83,6 +89,7 @@ func main() {
QueueClient: queueClient,
StorageService: storageSvc,
GatewayClient: gatewayClient,
GatewayLimit: gatewayLimit,
})
if err != nil {
appLogger.Fatal("初始化业务组件失败", zap.Error(err))

View File

@@ -40,12 +40,16 @@ func generateAdminDocs(outputPath string) error {
handlers.BusinessUserGroup = admin.NewBusinessUserGroupHandler(nil, nil)
handlers.ShopBusinessOwnerImport = admin.NewShopBusinessOwnerImportHandler(nil)
handlers.PhoneAssetAssociation = admin.NewPhoneAssetAssociationHandler(nil, nil)
// 网关限流 Handler 必须同时进入文档工厂,避免新增管理接口遗漏文档注册。
handlers.GatewayRateLimit = admin.NewGatewayRateLimitHandler(nil, nil, nil)
handlers.CarrierScheduler = admin.NewCarrierSchedulerHandler(nil, nil)
// 套餐真流量预警 Handler 必须同时进入文档工厂,避免新增管理接口遗漏文档注册。
handlers.PackageTrafficAlert = admin.NewPackageTrafficAlertHandler(nil, nil, nil, nil)
// 资产钱包自动续费配置 Handler 必须同时进入文档工厂,避免新增管理接口遗漏文档注册。
handlers.AssetAutoRenewal = admin.NewAssetAutoRenewalConfigHandler(nil, nil)
// 运营报表 Handler 必须同时进入文档工厂,避免新增管理接口遗漏文档注册。
handlers.OperationsReport = admin.NewOperationsReportHandler(nil, nil, nil)
handlers.StopResumeRule = admin.NewStopResumeRuleHandler(nil, nil, nil)
handlers.ClientPopup = apphandler.NewClientPopupHandler(nil, nil, nil)
handlers.H5PopupConfiguration = admin.NewH5PopupConfigurationHandler(nil, nil, nil)
// 企业微信 Handler 在此显式装配,避免新增管理接口遗漏文档注册。

View File

@@ -0,0 +1,270 @@
package main
import (
"context"
"encoding/json"
"os"
"strconv"
"strings"
"sync"
"time"
"go.uber.org/zap"
cardObservationApp "github.com/break/junhong_cmp_fiber/internal/application/cardobservation"
"github.com/break/junhong_cmp_fiber/internal/infrastructure/messaging/outbox"
"github.com/break/junhong_cmp_fiber/internal/infrastructure/nsqbus"
"github.com/break/junhong_cmp_fiber/internal/polling"
"github.com/break/junhong_cmp_fiber/internal/polling/carrierqueue"
"github.com/break/junhong_cmp_fiber/internal/task"
"github.com/break/junhong_cmp_fiber/pkg/config"
"github.com/break/junhong_cmp_fiber/pkg/constants"
"github.com/break/junhong_cmp_fiber/pkg/queue"
)
// carrierInitLockKey 新结构初始化锁:多实例时只由一个实例初始化。
const carrierInitLockKey = "polling:carrier:init_lock"
// carrierSchedulerRuntime 是按运营商账户调度在 Worker 内的运行组件:模式监听、初始化、分发器与 NSQ 消费者。
type carrierSchedulerRuntime struct {
cfg config.CarrierSchedulerConfig
runtime *workerRuntime
initializer *polling.CarrierInitializer
dispatcher *carrierqueue.Dispatcher
consumers *nsqbus.ConsumerManager
producer *nsqbus.Producer
alerts *carrierqueue.AlertWriter
executor *task.CarrierCardExecutor
singleton bool
instanceID string
logger *zap.Logger
mu sync.Mutex
lastMode string
reinitQueued bool
legacyRestarts func()
}
// newCarrierSchedulerParts 在 Worker 运行时初始化阶段创建调度参数、账户队列存储与模式接缝,
// 并注入旧调度相关组件(生命周期、轮询基类),使其在启动前即可按模式写入正确结构。
func newCarrierSchedulerParts(runtime *workerRuntime, appLogger *zap.Logger) {
settings := carrierqueue.NewDefaultSettingsProvider(runtime.db, runtime.redisClient, appLogger)
store := carrierqueue.NewStore(runtime.redisClient)
scheduling := polling.NewCarrierScheduling(store, settings, runtime.pollingConfigMgr, appLogger)
runtime.carrierSettings = settings
runtime.carrierStore = store
runtime.carrierScheduling = scheduling
runtime.lifecycleSvc.SetCarrierScheduling(scheduling)
runtime.pollingBase.SetSchedulerModeReader(func(ctx context.Context) string { return settings.Current(ctx).Mode })
if runtime.gatewayLimit != nil && runtime.gatewayLimit.Classifier != nil {
classifier := runtime.gatewayLimit.Classifier
runtime.pollingBase.SetErrorClassifier(func(ctx context.Context, op, message string) string {
return classifier.Classify(ctx, op, message).Category
})
}
}
// startCarrierScheduler 启动模式监听;分发器只在 leader/all 运行(多实例安全),消费者在所有处理任务的实例运行。
func startCarrierScheduler(ctx context.Context, cfg *config.Config, runtime *workerRuntime, steps queue.PollingStepHandlers,
singleton bool, legacyRestart func(), appLogger *zap.Logger) *carrierSchedulerRuntime {
carrierCfg := cfg.Polling.Carrier
instanceID := cfg.Worker.InstanceName
if instanceID == "" {
instanceID, _ = os.Hostname()
}
instanceID += "-" + strconv.Itoa(os.Getpid())
alerts := carrierqueue.NewAlertWriter(runtime.db, runtime.redisClient, runtime.carrierSettings, appLogger)
alerts.Start(ctx)
executor := task.NewCarrierCardExecutor(runtime.pollingBase, runtime.carrierStore, runtime.carrierSettings, alerts,
steps.Realname, steps.Carddata, steps.CardStatus, steps.Package, carrierCfg, instanceID)
c := &carrierSchedulerRuntime{
cfg: carrierCfg, runtime: runtime, singleton: singleton, instanceID: instanceID, logger: appLogger,
alerts: alerts, executor: executor, legacyRestarts: legacyRestart,
initializer: polling.NewCarrierInitializer(runtime.carrierScheduling, runtime.pollingIotCardStore, appLogger),
}
if carrierCfg.NSQDAddress != "" {
producer, err := nsqbus.NewProducer(carrierCfg.NSQDAddress, appLogger)
if err != nil {
appLogger.Error("创建 NSQ 生产者失败,按运营商账户调度不可用", zap.Error(err))
} else {
c.producer = producer
}
c.consumers = nsqbus.NewConsumerManager(nsqbus.ConsumerConfig{
NSQDAddress: carrierCfg.NSQDAddress, NSQLookupdAddresses: nsqbus.SplitAddresses(carrierCfg.NSQLookupdAddresses),
MsgTimeout: carrierCfg.MsgTimeout, MaxAttempts: carrierCfg.MaxAttempts,
}, constants.NSQChannelWorker, executor.Handle, appLogger)
}
var budgets carrierqueue.BudgetProvider
if runtime.gatewayLimit != nil && runtime.gatewayLimit.Limiter != nil {
budgets = runtime.gatewayLimit.Limiter
}
if c.producer != nil {
c.dispatcher = carrierqueue.NewDispatcher(runtime.db, runtime.carrierStore, runtime.carrierSettings, c.producer, budgets,
carrierCfg, instanceID, appLogger)
if singleton {
go c.dispatcher.Run(ctx)
}
}
go c.watch(ctx)
return c
}
// RequestReinit 轮询配置变更时请求按新配置重新初始化新结构(仅 carrier 模式生效)。
func (c *carrierSchedulerRuntime) RequestReinit() {
if c == nil {
return
}
c.mu.Lock()
c.reinitQueued = true
c.mu.Unlock()
}
func (c *carrierSchedulerRuntime) watch(ctx context.Context) {
ticker := time.NewTicker(3 * time.Second)
defer ticker.Stop()
for {
c.step(ctx)
select {
case <-ctx.Done():
c.shutdown()
return
case <-ticker.C:
}
}
}
func (c *carrierSchedulerRuntime) step(ctx context.Context) {
defer func() {
if r := recover(); r != nil {
c.logger.Error("按运营商账户调度模式监听 panic,已恢复", zap.Any("panic", r))
}
}()
mode := c.runtime.carrierSettings.Current(ctx).Mode
c.mu.Lock()
previous := c.lastMode
c.lastMode = mode
reinit := c.reinitQueued
c.reinitQueued = false
c.mu.Unlock()
if mode == constants.PollingSchedulerModeCarrier {
if previous != mode {
c.logger.Info("卡轮询调度模式:按运营商账户调度", zap.String("previous", previous))
}
if c.singleton {
c.ensureInitialized(ctx, reinit)
}
c.syncConsumers(ctx)
return
}
if previous == constants.PollingSchedulerModeCarrier {
c.logger.Info("卡轮询调度模式回到旧分片调度:停止领取与消费,重建旧分片队列")
if c.consumers != nil {
c.consumers.StopAll(c.grace())
c.consumers.Reset()
}
if c.singleton {
c.runtime.redisClient.Del(ctx, constants.RedisCarrierSchedulerInitStateKey)
if c.legacyRestarts != nil {
c.legacyRestarts()
}
}
}
}
// ensureInitialized 切到 carrier 后初始化新结构;初始化完成前分发器不领取。多实例时用初始化锁协调。
func (c *carrierSchedulerRuntime) ensureInitialized(ctx context.Context, reinit bool) {
redisClient := c.runtime.redisClient
state, _ := redisClient.Get(ctx, constants.RedisCarrierSchedulerInitStateKey).Result()
if state == carrierqueue.InitStateDone && !reinit {
return
}
ok, err := redisClient.SetNX(ctx, carrierInitLockKey, c.instanceID, 30*time.Minute).Result()
if err != nil || !ok {
return
}
defer redisClient.Del(context.WithoutCancel(ctx), carrierInitLockKey)
if state != carrierqueue.InitStateDone {
redisClient.Set(ctx, constants.RedisCarrierSchedulerInitStateKey, carrierqueue.InitStateRunning, 0)
}
started := time.Now()
written, initErr := c.initializer.Run(ctx)
if initErr != nil {
c.logger.Error("按运营商账户调度初始化失败,下一轮重试", zap.Error(initErr))
return
}
redisClient.Set(ctx, constants.RedisCarrierSchedulerInitStateKey, carrierqueue.InitStateDone, 0)
c.logger.Info("按运营商账户调度初始化完成,开始领取", zap.Int64("cards", written), zap.Duration("duration", time.Since(started)),
zap.String("metric", "polling.carrier.initialized"))
}
// syncConsumers 按当前运营商账户订阅卡任务 topic,max_in_flight 取账户调度参数。
func (c *carrierSchedulerRuntime) syncConsumers(ctx context.Context) {
if c.consumers == nil || c.dispatcher == nil {
return
}
desired := map[string]int{}
for _, carrierID := range c.dispatcher.CarrierIDs(ctx) {
desired[constants.NSQCardTopic(carrierID)] = c.dispatcher.ParamsFor(carrierID).CardMaxInFlight
}
c.consumers.Sync(desired)
}
func (c *carrierSchedulerRuntime) grace() time.Duration {
if c.cfg.ShutdownGrace > 0 {
return c.cfg.ShutdownGrace
}
return 30 * time.Second
}
// shutdown 优雅退出:停止订阅,宽限期内等待执行中的卡任务完成;未完成的由租约到期后重新领取。
func (c *carrierSchedulerRuntime) shutdown() {
if c.consumers != nil {
c.consumers.StopAll(c.grace())
}
if c.producer != nil {
c.producer.Stop()
}
}
// InitProgress 返回新结构初始化进度。
func (c *carrierSchedulerRuntime) InitProgress() polling.InitProgress {
return c.initializer.Progress()
}
// carrierAwareSeriesConsumer 是业务事件观测序列 Outbox 消费者的模式接缝:按运营商账户调度时把业务事件改为按场景请求检查
// (事件在原业务事务中写入,消费即在事务提交之后),旧分片调度时沿用原观测序列。
type carrierAwareSeriesConsumer struct {
legacy outbox.EventConsumer
requester *polling.CheckRequester
}
// Consume 消费业务观测序列请求事件。
func (c *carrierAwareSeriesConsumer) Consume(ctx context.Context, envelope outbox.DeliveryEnvelope) error {
if c.requester == nil || !c.requester.Active(ctx) {
return c.legacy.Consume(ctx, envelope)
}
var event cardObservationApp.SeriesRequestedEvent
if err := json.Unmarshal(envelope.Payload, &event); err != nil {
return nil
}
cardIDs := []uint{event.ResourceID}
if event.ResourceType == constants.CardObservationResourceTypeDevice {
cardIDs = event.ResourceIDs
}
c.requester.RequestCards(ctx, businessEventCheckScene(event), cardIDs, event.ExpectedValue)
return nil
}
// businessEventCheckScene 把业务观测事件映射为请求检查场景:订单支付、套餐激活、自动续费购包按事件来源区分,停复机成功按观测场景。
func businessEventCheckScene(event cardObservationApp.SeriesRequestedEvent) string {
switch {
case strings.HasPrefix(event.EventID, "card-observation:order:"):
return constants.PollingCheckSceneOrderPaid
case strings.HasPrefix(event.EventID, "card-observation:auto-purchase:"), strings.HasPrefix(event.EventID, "asset-auto-renewal:"):
return constants.PollingCheckSceneAutoRenewalPurchase
case event.Scene == constants.CardObservationSceneBusinessStop || event.Scene == constants.CardObservationSceneBusinessResume:
return constants.PollingCheckSceneStopResumeSucceeded
default:
return constants.PollingCheckScenePackageActivated
}
}

View File

@@ -34,6 +34,7 @@ import (
cardObservationInfra "github.com/break/junhong_cmp_fiber/internal/infrastructure/cardobservation"
carrierThresholdInfra "github.com/break/junhong_cmp_fiber/internal/infrastructure/carrierthreshold"
commissionDelivery "github.com/break/junhong_cmp_fiber/internal/infrastructure/commissiondelivery"
"github.com/break/junhong_cmp_fiber/internal/infrastructure/gatewaylimit"
"github.com/break/junhong_cmp_fiber/internal/infrastructure/integrationlog"
"github.com/break/junhong_cmp_fiber/internal/infrastructure/messaging/outbox"
notificationInfra "github.com/break/junhong_cmp_fiber/internal/infrastructure/notification"
@@ -44,6 +45,7 @@ import (
wecomInfra "github.com/break/junhong_cmp_fiber/internal/infrastructure/wecom"
"github.com/break/junhong_cmp_fiber/internal/model"
"github.com/break/junhong_cmp_fiber/internal/polling"
"github.com/break/junhong_cmp_fiber/internal/polling/carrierqueue"
iot_card_svc "github.com/break/junhong_cmp_fiber/internal/service/iot_card"
pollingSvc "github.com/break/junhong_cmp_fiber/internal/service/polling"
refundSvc "github.com/break/junhong_cmp_fiber/internal/service/refund"
@@ -99,6 +101,14 @@ type workerRuntime struct {
carrierThresholdService *carrierThresholdApp.Service
// assetAutoRenewalService 是资产钱包自动续费的每日扫描、复机消费者与恢复扫描的唯一用例实例。
assetAutoRenewalService *assetAutoRenewalApp.Service
// gatewayLimit 是卡网关运营商账户限流组件(限流、统计、停复机记录)。
gatewayLimit *gatewaylimit.Runtime
// 按运营商账户调度:调度参数、账户队列存储与模式接缝(模式为 carrier 时各入口写新结构)。
carrierSettings *carrierqueue.SettingsProvider
carrierStore *carrierqueue.Store
carrierScheduling *polling.CarrierScheduling
// carrierReinit 在 carrier 模式下轮询配置变更时请求重新初始化新结构。
carrierReinit func()
}
func main() {
@@ -168,6 +178,12 @@ func runWorker(cfg *config.Config) {
taskHandler := createTaskHandler(runtime, appLogger)
taskHandler.RegisterHandlers()
carrierRuntime := startCarrierScheduler(ctx, cfg, runtime, taskHandler.PollingSteps(), runsSingletonModules(cfg.Worker.Role), func() {
if pollingInitializer != nil {
pollingInitializer.Restart(ctx)
}
}, appLogger)
runtime.carrierReinit = carrierRuntime.RequestReinit
registerWeComApprovalTasks(taskHandler.GetMux(), runtime, cfg, appLogger)
registerAgentRechargeRecoveryTask(taskHandler.GetMux(), runtime, appLogger)
registerRefundChannelRecoveryTask(taskHandler.GetMux(), runtime, appLogger)
@@ -178,6 +194,12 @@ func runWorker(cfg *config.Config) {
registerAuditArchiveTask(taskHandler.GetMux(), runtime, cfg.Worker.AuditRetentionCleanupEnabled, cfg.Worker.AuditArchiveTasksEnabled, appLogger, retentionLogger)
outboxHandler := outbox.NewHandler(runtime.outboxConsumers)
taskHandler.GetMux().HandleFunc(constants.TaskTypeOutboxDeliver, outboxHandler.Handle)
partitionMaintainer := gatewaylimit.NewPartitionMaintainer(postgres.NewCardStopResumeRecordStore(runtime.db), runtime.gatewayLimit.Settings, appLogger)
taskHandler.GetMux().HandleFunc(constants.TaskTypeStopResumeRecordPartition, partitionMaintainer.Handle)
dueRecalculator := polling.NewDueRecalculator(runtime.db, runtime.redisClient, runtime.pollingConfigMgr, runtime.pollingQueueMgr, appLogger)
dueRecalculator.SetCarrierScheduling(runtime.carrierScheduling)
recalcHandler := polling.NewRecalculateTaskHandler(dueRecalculator, runtime.redisClient, appLogger)
taskHandler.GetMux().HandleFunc(constants.TaskTypePollingRecalculateDue, recalcHandler.Handle)
startOutboxRelay(ctx, runtime, cfg.Worker.InstanceName, appLogger)
rescuePendingImportTasks(ctx, runtime, appLogger)
@@ -266,6 +288,11 @@ func initWorkerRuntime(ctx context.Context, cfg *config.Config, appLogger *zap.L
storageSvc := initStorage(cfg, appLogger)
gatewayClient := initGateway(cfg, appLogger)
// 卡网关调用按本系统运营商账户限流与统计(默认只统计不拦截)。
gatewayLimit := gatewaylimit.NewRuntime(db, redisClient, appLogger)
if gatewayClient != nil {
gatewayClient.SetCallGuard(gatewayLimit.Guard)
}
asynqClient := asynq.NewClient(asynq.RedisClientOpt{
Addr: redisAddr,
@@ -282,6 +309,7 @@ func initWorkerRuntime(ctx context.Context, cfg *config.Config, appLogger *zap.L
QueueClient: workerQueueClient,
StorageService: storageSvc,
GatewayClient: gatewayClient,
GatewayLimit: gatewayLimit,
}
workerResult, err := bootstrap.BootstrapWorker(workerDeps)
@@ -289,11 +317,19 @@ func initWorkerRuntime(ctx context.Context, cfg *config.Config, appLogger *zap.L
appLogger.Fatal("Worker Bootstrap 失败", zap.Error(err))
}
workerServer := queue.NewServer(redisClient, &cfg.Queue, appLogger)
workerServer := queue.NewServer(redisClient, &cfg.Queue, appLogger, queue.WithRateLimitedRetry(queue.RateLimitedRetry{
Match: gateway.IsRateLimited,
Delay: func() time.Duration {
return gatewayLimit.Settings.Current(context.Background()).RetryDelay
},
OnDeferred: gatewayLimit.TrackDeferral,
}))
outboxQueueClient := workerQueueClient
pollingConfigStore := postgres.NewPollingConfigStore(db)
pollingConfigMgr := polling.NewPollingConfigManager(pollingConfigStore, redisClient, appLogger)
pollingConfigMgr.SetTierResolver(polling.NewTierResolver(postgres.NewPackageUsageStore(db, redisClient),
postgres.NewDeviceSimBindingStore(db, redisClient), postgres.NewCarrierStore(db), appLogger))
if err := pollingConfigMgr.Load(ctx); err != nil {
appLogger.Warn("加载轮询配置失败,使用空配置继续", zap.Error(err))
}
@@ -324,6 +360,9 @@ func initWorkerRuntime(ctx context.Context, cfg *config.Config, appLogger *zap.L
cfg.Polling.VerboseLog,
cfg.Worker.PollingTotalMaxConcurrency,
)
pollingBase.SetRateLimitedDelay(func() time.Duration {
return gatewayLimit.Settings.Current(context.Background()).RetryDelay
})
if stopResumeSvc, ok := workerResult.Services.StopResumeService.(*iot_card_svc.StopResumeService); ok {
stopResumeSvc.SetPollingCallback(lifecycleSvc)
}
@@ -337,6 +376,7 @@ func initWorkerRuntime(ctx context.Context, cfg *config.Config, appLogger *zap.L
db: db,
storageSvc: storageSvc,
gatewayClient: gatewayClient,
gatewayLimit: gatewayLimit,
asynqClient: asynqClient,
workerResult: workerResult,
workerServer: workerServer,
@@ -352,6 +392,7 @@ func initWorkerRuntime(ctx context.Context, cfg *config.Config, appLogger *zap.L
// 自动续费的复机执行同样复用既有停复机单一事实源。
assetAutoRenewalService: workerResult.Services.AssetAutoRenewal,
}
newCarrierSchedulerParts(runtime, appLogger)
registerNotificationOutboxConsumer(runtime, appLogger)
registerWalletOutboxConsumer(runtime, appLogger)
registerCardObservationOutboxConsumer(runtime, appLogger)
@@ -685,7 +726,10 @@ func registerCardObservationOutboxConsumer(runtime *workerRuntime, appLogger *za
queue.NewCardObservationSeriesScheduler(runtime.outboxQueueClient),
cardObservationInfra.NewSeriesAttemptLogger(integrationlog.NewRepository(runtime.db)),
)
seriesConsumer := cardObservationInfra.NewSeriesRequestedConsumer(seriesTrigger)
seriesConsumer := &carrierAwareSeriesConsumer{
legacy: cardObservationInfra.NewSeriesRequestedConsumer(seriesTrigger),
requester: polling.NewCheckRequester(runtime.carrierScheduling, runtime.db, appLogger),
}
if err := runtime.outboxConsumers.Register(constants.OutboxEventTypeCardSeriesRequested, seriesConsumer); err != nil {
appLogger.Fatal("注册业务卡观测序列 Outbox 消费者失败",
zap.String("event_type", constants.OutboxEventTypeCardSeriesRequested), zap.Error(err))
@@ -815,6 +859,13 @@ func startPollingInitializer(ctx context.Context, runtime *workerRuntime, appLog
pollingInitializer.StartBackground(ctx)
runtime.pollingConfigMgr.WatchChanges(ctx, func(hadConfigs, hasConfigs bool) {
if runtime.carrierScheduling.Active(ctx) {
// 按运营商账户调度时按新配置重新初始化新结构,旧分片队列在回退时再重建。
if runtime.carrierReinit != nil {
runtime.carrierReinit()
}
return
}
if hasConfigs {
appLogger.Info("轮询配置已变更,触发队列重新初始化",
zap.Bool("had_configs", hadConfigs))
@@ -867,6 +918,7 @@ func startPollingScheduler(
dataResetHandler,
)
pollingScheduler.SetInitializer(pollingInitializer)
pollingScheduler.SetCarrierScheduling(runtime.carrierScheduling)
if err := pollingScheduler.Start(ctx); err != nil {
appLogger.Error("启动轮询调度器失败", zap.Error(err))
@@ -996,6 +1048,14 @@ func registerAsynqScheduleTasks(asynqScheduler *asynq.Scheduler, auditArchiveEna
)); err != nil {
return fmt.Errorf("注册数据清理定时任务失败: %w", err)
}
if _, err := asynqScheduler.Register("CRON_TZ=Asia/Shanghai 30 2 * * *", asynq.NewTask(
constants.TaskTypeStopResumeRecordPartition,
nil,
asynq.MaxRetry(3),
asynq.Queue(constants.QueueDataCleanup),
)); err != nil {
return fmt.Errorf("注册停复机记录分区维护定时任务失败: %w", err)
}
if _, err := asynqScheduler.Register("15 2 * * *", asynq.NewTask(
constants.TaskTypeNotificationCleanup,
nil,

View File

@@ -77,6 +77,10 @@ services:
- JUNHONG_WECOM_TIMEOUT=10s
- JUNHONG_APPROVAL_LEGACY_REFUND_MANUAL_ENABLED=${JUNHONG_APPROVAL_LEGACY_REFUND_MANUAL_ENABLED:-true}
- JUNHONG_APPROVAL_LEGACY_OFFLINE_RECHARGE_PAY_ENABLED=${JUNHONG_APPROVAL_LEGACY_OFFLINE_RECHARGE_PAY_ENABLED:-true}
# 卡流量日快照计量(API 与 Worker 必须一致;默认与未配置时行为不变)
- JUNHONG_TRAFFIC_CALC_MODE=${JUNHONG_TRAFFIC_CALC_MODE:-off}
- JUNHONG_TRAFFIC_CALC_CARDS=${JUNHONG_TRAFFIC_CALC_CARDS:-}
- JUNHONG_TRAFFIC_FLOW_SNAPSHOT_ENABLED=${JUNHONG_TRAFFIC_FLOW_SNAPSHOT_ENABLED:-true}
# 短信服务配置
- JUNHONG_SMS_GATEWAY_URL=https://gateway.sms.whjhft.com:8443
- JUNHONG_SMS_USERNAME=JH0001
@@ -143,9 +147,16 @@ services:
# 七月迭代:单 Worker 测试环境同时承担调度和消费
- JUNHONG_WORKER_ROLE=${JUNHONG_WORKER_ROLE:-all}
- JUNHONG_WORKER_INSTANCE_NAME=${JUNHONG_WORKER_INSTANCE_NAME:-test-worker-all-1}
# 卡流量日快照计量(API 与 Worker 必须一致;默认与未配置时行为不变)
- JUNHONG_TRAFFIC_CALC_MODE=${JUNHONG_TRAFFIC_CALC_MODE:-off}
- JUNHONG_TRAFFIC_CALC_CARDS=${JUNHONG_TRAFFIC_CALC_CARDS:-}
- JUNHONG_TRAFFIC_FLOW_SNAPSHOT_ENABLED=${JUNHONG_TRAFFIC_FLOW_SNAPSHOT_ENABLED:-true}
# Worker 执行企微审批提交、同步和恢复任务
- JUNHONG_WECOM_BASE_URL=https://qyapi.weixin.qq.com
- JUNHONG_WECOM_TIMEOUT=10s
# 按运营商账户调度的 NSQ(卡轮询分发;调度模式在受控系统配置 polling.scheduler.mode 切换)
- JUNHONG_POLLING_CARRIER_NSQD_ADDRESS=${JUNHONG_POLLING_CARRIER_NSQD_ADDRESS:-nsqd:4150}
- JUNHONG_POLLING_CARRIER_NSQLOOKUPD_ADDRESSES=${JUNHONG_POLLING_CARRIER_NSQLOOKUPD_ADDRESSES:-nsqlookupd:4161}
volumes:
- ./logs:/app/logs
networks:
@@ -153,6 +164,39 @@ services:
depends_on:
api:
condition: service_healthy
nsqd:
condition: service_started
logging:
driver: "json-file"
options:
max-size: "10m"
max-file: "3"
# NSQ:只承担卡轮询的执行分发,Redis 是唯一权威状态(add-carrier-polling-scheduler)
nsqlookupd:
image: nsqio/nsq:v1.3.0
container_name: junhong-cmp-nsqlookupd
restart: unless-stopped
command: /nsqlookupd
networks:
- junhong-network
logging:
driver: "json-file"
options:
max-size: "10m"
max-file: "3"
nsqd:
image: nsqio/nsq:v1.3.0
container_name: junhong-cmp-nsqd
restart: unless-stopped
command: /nsqd --lookupd-tcp-address=nsqlookupd:4160 --broadcast-address=nsqd --data-path=/data --mem-queue-size=10000 --max-msg-timeout=15m
volumes:
- ./nsq-data:/data
networks:
- junhong-network
depends_on:
- nsqlookupd
logging:
driver: "json-file"
options:

View File

@@ -283,9 +283,9 @@
- **最小正例**:外部失败写脱敏 Integration Log,HTTP 请求写 Access Log。
- **最小反例**:把完整凭证写 app.log。
- **机械检查/人工原因**:人工核对 Logger 字段、Sanitizer 与事实存储;本地按 request_id/correlation_id 查询日志。
- **例外条件**:第三方要求回显的非敏感渠道码可保留。
- **例外条件**:第三方要求回显的非敏感渠道码可保留。卡网关调用(add-carrier-gateway-rate-limit):普通轮询查询不写 Integration Log,失败原因与调用量由 Redis 按运营商账户的分钟计数与卡最近错误承载(同时是优先执行的轮次仍写);系统自动停复机不写 Integration Log 与 Audit Event,每次上游调用写停复机记录表 `tb_card_stop_resume_record`;后台人工点击、通道阈值、自动续费及账户、个人客户、OpenAPI 发起的停复机保持原写法。
- **Owner**:审计与安全负责人
- **最后验证日期**:2026-08-07
- **最后验证日期**:2026-10-24
- **更新触发条件**:日志字段、保留策略或外部集成变化
## ENG-SECRET-001

File diff suppressed because it is too large Load Diff

View File

@@ -5,7 +5,8 @@
| 日期 | 2026-10-20 |
| 状态 | 方案,待评审,**代码尚未修改** |
| 思路 | **流量怎么算,照搬老卡管;套餐那边一行不动**:只替换"增量从哪来",增量照旧通过 V1 事件交给现有 `DeductDataUsage` 扣减 |
| 改动范围 | 观测链路(算增量)+ 新增卡日快照表 + 日流量读取。**套餐表、套餐服务、激活/重置/改已用量/换货/退款入口、扣减顺序、停复机全部不改** |
| 修订 | 第 2 版(2026-10-20):对照反编译结果与新卡管代码复核后修订——增加切换水位(4.6)、增量改为按月对账(4.5)、已切换卡的通道阈值判定(4.5)、去掉"读数是否有效"标记、更正对老卡管的几处描述;新增第 10 章停复机规则与实名管控;已切换卡的卡表本月用量改由快照覆盖(4.5),切换与退回要求全部停止后再启动(5.2) |
| 改动范围 | 流量部分(第 1~9 章):观测链路(算增量)+ 新增卡日快照表 + 切换水位表 + 日流量读取 + 已切换卡的通道阈值判定条件;**套餐表、套餐服务、V1 消费者、激活/重置/改已用量/换货/退款入口、扣减顺序、停复机全部不改**。停复机部分(第 10 章):停机与复机只看套餐和流量(通道开启实名管控时加上实名)、复机不看停机原因、实名默认不管控、暂停轮询的卡不自动停复机;独立开发、独立上线 |
| 与旧方案关系 | 替代 `docs/流量事实模型技术方案.md`(第 5 版)的计量部分 |
| 老卡管依据 | 反编译 `kyhl-api/WEB-INF/lib` 下 `kyhl-service-1.0.jar`、`kyhl-dal-1.0.jar`、`kyhl-common-1.0.jar`、`kyhl-common-jdbk-1.0.jar` |
@@ -17,10 +18,12 @@
2. **老卡管不依赖基线**:每张卡每天存一条"本月累计"快照,任何一段时间的用量都用快照现算(区间内最大值 − 区间开始前的最大值)。它没有"读数下降"的概念,已经稳定运行多年。
3. **本方案**:
- 流量计算照搬老卡管:日快照、按各运营商的重置日(`tb_carrier.data_reset_day`)换算成自然月、重置日防旧值、按区间取最大值做差。
- 每次观测的增量 = **今天的快照用量 − 今天已经发出的增量**,不再用卡表基线。
- 每次观测的增量 = **本月最大快照 − 本月已结算合计**(按月对账),不再用卡表基线。
- 卡第一次走新路径时记一条**切换水位**:本次仍按老路径的增量发,之前的部分视为老路径已结算。切换衔接、补数据截断、冻结卡判定都以它为准。
- 增量照旧写进 V1 事件 `CardTrafficIncremented`,由现有消费者调用 `DeductDataUsage` 扣减。**套餐分配、加油包顺序、虚流量阈值、停复机、流量详单全部沿用现有代码。**
4. **二次销售**:卡没有生效套餐时,增量照样计算,但**只推进"已发出"标记、不发事件**。所以激活之前的用量永远不会扣到新套餐上,新套餐只扣激活之后的增长。
5. **先做卡流量计算,补数据放到后期**:新方案上线后,冻结卡从切换起恢复正常计量;切换前少扣的部分由后期的独立脚本补(5.5)。补数据依据的是运营商当前读数,只能补执行时所在运营商周期内的部分,更早的补不回来。
- 已切换的卡,通道阈值按"读数有效且未被防旧值处理"判定,不再受卡表基线影响,冻结卡切换后通道达量停机随即恢复。
4. **二次销售**:卡没有生效套餐时,增量照样计算,但**只记为已结算、不发事件**。激活之前被采到的用量不会扣到新套餐上;精度到观测粒度:激活后第一次观测里,"上次观测到激活"这一段仍会算给新套餐。
5. **先做卡流量计算,补数据放到后期**:新方案上线后,冻结卡从切换起恢复正常计量;切换前少扣的部分由后期的独立脚本补(5.5)。补数据依据运营商读数(未切换的卡现场查询,已切换的卡取切换水位里切换时的读数),只能补执行时所在运营商周期内的部分,更早的补不回来。
### 1.1 已定决策(2026-10-20)
@@ -29,12 +32,17 @@
| D1 | 流量计算 | 照搬老卡管:日快照、写入时按运营商重置日(`tb_carrier.data_reset_day`,任意 1~28 号)换算成自然月、按区间取最大值做差,没有读数下降判定。重置日不写死,运营商配置改了随之生效 |
| D2 | 套餐侧 | 一行不改:增量照旧走 V1 事件与 `DeductDataUsage` |
| D3 | 重置日防旧值 | **完全照搬老卡管**(含比较基准):只在重置日当天判断,`R = 1` 比上月最大快照、`R > 1` 比本月 1~R−1 号最大快照,超过 0.9 倍按 0 记;第二天起照常累加,不挂起、不告警。老卡管多年运行没有暴露问题 |
| D4 | 卡无生效套餐时的增量 | 只推进 `emitted_mb`,不发事件,用量丢弃 |
| D4 | 卡无生效套餐时的增量 | 只记入 `settled_mb`(已结算),不发事件,用量丢弃 |
| D5 | 快照记录范围 | 全部被采集的卡 |
| D6 | 上线方式 | 按名单灰度,单位是扣减载体,设备整台切;第一批为 103154 所在设备 15276,核对后扩批再全量。非灰度卡行为与现在一致(4.8) |
| D7 | 积压的无套餐回执 | 业务后续手动调整,不在本方案内;处理前不得重放 |
| D8 | 顺序 | 先上卡流量计算,补数据放后期(5.5) |
| D9 | 补数据:本周期中间重置过套餐的冻结卡 | 不自动补,进清单,附按天数比例的估算值,由人工决定 |
| D10 | 增量口径 | 按月对账:`Δ = 本月最大快照 − 本月已结算合计`,不按天对账(4.5) |
| D11 | 切换衔接 | 记切换水位:卡第一次走新路径时,本次照发老路径增量,之前的部分记为已结算;不依赖阶段 1 是否覆盖整月(4.6) |
| D12 | 已切换卡的通道阈值 | 读数有效且本次读数未被防旧值按 0 处理即判定,不再看老路径是否接受读数(4.5) |
| D13 | 套餐在扣减前刚好用完 | 消费者不改,写入已知限制并监控(第 9 节) |
| D14~D17 | 停复机规则、实名管控、暂停轮询 | 见 10.3 |
## 2. 现在的漏洞(已验证,详见旧方案第 2 节)
@@ -45,18 +53,21 @@
| 3 | **无效响应按 0 处理**:`used` 为空、null、无法解析时当成 0,进而触发第 2 条 | `internal/gateway/models.go:55-78` |
| 4 | **二次销售多计**:新用户激活时,卡上的基线如果不是激活那一刻的值,差额就扣到了新套餐上 | 资产回收换代见 `docs/资产回收换代技术方案.md` |
| 5 | **日流量从来没存成功**:`tb_card_daily_usage` 的表结构和代码不一致,表里 0 行 | 旧方案 P14 |
| 6 | **无套餐期间的增量会被扣到下一个套餐上**:老路径不管卡有没有套餐都发 V1 事件;消费者调 `DeductDataUsage` 遇到"没有可用套餐"直接返回错误,任务由 Asynq 重试,重试期间套餐一激活,积压的增量就被扣到新套餐上——这是二次销售多计的另一条路径 | `application/cardobservation/traffic.go:73-76` 无套餐判断;`infrastructure/cardobservation/event.go:276-277` 报错即返回;`usage_service.go:110`。**生产核实(2026-10-20)**:回执停在"处理中"(扣减未完成)的有 18,979 条、增量约 11.4 TB;其中 14,838 条对应的卡当前没有生效套餐,**4,141 条(2,404 张卡)对应的卡当前有生效套餐**,这批任务若被重试或人工重放,会扣到现在的套餐上,不得重放。**积压部分由业务后续手动调整,本方案只保证新增的不再产生** |
| 6 | **无套餐期间的增量会被扣到下一个套餐上**:老路径不管卡有没有套餐都发 V1 事件;消费者调 `DeductDataUsage` 遇到"没有可用套餐"直接返回错误,任务由 Asynq 重试,重试期间套餐一激活,积压的增量就被扣到新套餐上——这是二次销售多计的另一条路径 | `application/cardobservation/traffic.go:73-76` 不判断有无套餐,有正增量就发事件;`infrastructure/cardobservation/event.go:276-277` 报错即返回;`usage_service.go:110`。**生产核实(2026-10-20)**:回执停在"处理中"(扣减未完成)的有 18,979 条、增量约 11.4 TB;其中 14,838 条对应的卡当前没有生效套餐,**4,141 条(2,404 张卡)对应的卡当前有生效套餐**,这批任务若被重试或人工重放,会扣到现在的套餐上,不得重放。**积压部分由业务后续手动调整,本方案只保证新增的不再产生** |
## 3. 老卡管怎么算(反编译核实)
### 3.1 日快照:每卡每天一条,存"本月累计"
出处:`YidongDayFlowService.saveYidongDayFlow`、`PtLiantongDayFlowService.savePtLiantongDayFlow`。
出处:`YidongDayFlowService.saveYidongDayFlow`、`PtLiantongDayFlowService.saveLiantongDayFlow`。
- 表 `tbl_*_day_flow`:`iccid_mark`、`flow_date`、`used_flow`(自然月累计 MB);普联通另存 `vendor_used_flow`(运营商原始读数)。
- 同一天多次查询:用新值覆盖。
- 移动:新值比旧值小就跳过(每月 1 号除外)。
- 普联通:读数 ≤ 0 不覆盖;当天第一条就是 0 时,沿用昨天的值。
- 普联通:读数 ≤ 0 不覆盖;当天第一条就是 0 时,沿用昨天的值;其余情况后写覆盖,可以变小。
- 电信、广电、广电物联网、5G:与移动、普联通同类(1 号防旧值 + 同一天覆盖),细节略有差异。
- 老卡管按 category 分派到各自的表(`CardFlowUtils.getCardFlow`):电信 1、移动 3、联通 4/8/9、5G 108、广电 112/113 等。设备是主卡、副卡各自按区间取值后相加(`CardFlowUtils.getTraffic`)。
- 电信采集时按"本月 1 号~今天"向运营商查询(`TelecomUtilNew.queryThisMonthDianxinDayFlow`),拿到的是自然月累计,之后同样写日快照、按快照计费。
### 3.2 联通:写入时把运营商周期换算成自然月
@@ -67,14 +78,16 @@
| 27 号~月底 | 运营商读数 + 本月 1~26 号的最大快照 |
| 1~26 号 | 运营商读数 − 上月 27 号~月底这段的用量(= 上月最大快照 − 上月 27 号前的最大快照) |
> 老代码第二行取"上月 27 号前"时用的是 `< 26 号`,少算了一天,本方案按"1 号~重置日前一天"修正。
> 核实:老代码取"上月 1~26 号最大快照"用的是 `findFlowByBeforeDate`,SQL 为 `flow_date >= 1号 and flow_date <= 26号`,包含 26 号,取值正确;27 号起取"本月 1~26 号最大快照"用的是 `getMaxUsedFlow`,SQL 为 `flow_date < 27号`,同样正确。
> 老代码的小缺陷:`if ((flow -= subFlow) < 0) subFlow = 0` 改的是 `subFlow` 而不是 `flow`,负值会写进快照,靠查询时 `used_flow > 0` 过滤掉。本方案直接按 0 处理。
### 3.3 重置日防旧值:只有一条规则
| 运营商 | 规则 |
|---|---|
| 移动(1 号清零) | 每月 1 号,读数 > 上月最大快照 × 0.9 → 记为 0 |
| 普联通(27 号清零) | 每月 27 号,读数 > 本月 1~26 号最大快照 × 0.9 → 运营商读数按 0 计 |
| 移动、电信、广电、广电物联网、5G(1 号清零) | 每月 1 号,读数 > 上月最大快照 × 0.9 → 记为 0 |
| 普联通 category 9(27 号清零) | 每月 27 号,读数 > 本月 1~26 号最大快照 × 0.9 → 运营商读数按 0 计(代码写作 `appendFlow > 0 && flow > appendFlow × 1.9`,`flow` 已含 `appendFlow`) |
| 联通 category 4、8 | 没有防旧值,只做自然月换算 |
意思是:重置日读到的值还接近上个周期的总量,就认为运营商还没清零或者读到了旧缓存,当天先不增长。除此之外,老卡管**没有任何读数下降的判定**。
@@ -100,13 +113,13 @@
| # | 原则 |
|---|---|
| R1 | **流量照搬老卡管**:日快照 + 自然月换算 + 重置日防旧值 + 区间取最大值做差 |
| R2 | **套餐侧零改动**:增量照旧走 V1 事件和 `DeductDataUsage`;套餐表、套餐入口、分配顺序、停复机都不改 |
| R2 | **套餐侧零改动**:增量照旧走 V1 事件和 `DeductDataUsage`;套餐表、套餐入口、分配顺序、停复机都不改(停复机的调整见第 10 章,与流量部分无关) |
| R3 | **不再用卡表基线算增量**:增量只由快照得出;卡表三列照旧由老逻辑维护,只作兼容展示 |
| R4 | **日期一律按 `Asia/Shanghai`** |
### 4.2 数据模型
只新增一张表,**不改任何现有表**。迁移从当前最新编号之后开始,成对 up/down。
新增两张表,**不改任何现有表**。迁移从当前最新编号之后开始,成对 up/down。
**`tb_card_flow_day`(卡日快照,永久保留;仓库现无分区表,一年约 3,200 万行,靠唯一索引足够,本期不分区)**
@@ -117,13 +130,32 @@
| `flow_date` | date | 上海日期 |
| `month_used_mb` | numeric(14,3) | 自然月累计(换算后),计算用 |
| `vendor_used_mb` | numeric(14,3) | 当天最后一次有效的运营商原始读数,排障用 |
| `emitted_mb` | numeric(14,3) | 当天已经发出的增量合计(见 4.5),默认 0 |
| `settled_mb` | numeric(14,3) | 当天已结算的增量合计,默认 0。包括:发出 V1 的增量、无生效套餐时丢弃的增量、切换时记为"老路径已结算"的部分(见 4.5、4.6) |
| `stale_guarded` | boolean | 重置日当天是否触发过防旧值(读数按 0 记) |
| `obs_count` | int | 当天有效观测次数 |
| `created_at`、`updated_at` | timestamptz | |
唯一键 `(iot_card_id, flow_date)`。每卡每天一行,8.9 万卡约 3200 万行/年。
**`tb_card_flow_switch`(切换水位,每卡一行)**
卡第一次走新路径时写入(4.6),之后只读。退回老路径时删除(5.2)。
| 列 | 类型 | 含义 |
|---|---|---|
| `id` | bigserial | |
| `iot_card_id` | bigint | 唯一 |
| `switched_at` | timestamptz | 第一次走新路径的观测时刻 |
| `month_used_mb` | numeric(14,3) | 切换时本月最大快照(写入本次读数后) |
| `vendor_used_mb` | numeric(14,3) | 切换时的运营商读数 `r` |
| `old_baseline_mb` | numeric(14,3) | 切换时卡表 `last_gateway_reading_mb` |
| `old_month_usage_mb` | numeric(14,3) | 切换时卡表 `current_month_usage_mb` |
| `old_reading_accepted` | boolean | 切换这次观测老路径是否接受读数(冻结卡为 false) |
| `old_increment_mb` | numeric(14,3) | 切换这次观测老路径算出的增量(照此发 V1) |
| `created_at` | timestamptz | |
作用:切换衔接(4.6)、补数据截断与冻结卡判定(5.5)。
### 4.3 写快照(照搬 3.1~3.3,按运营商重置日 `R` 统一)
`R` 取 `tb_carrier.data_reset_day`。**运营商在每月 `R` 号 0 点(北京时间,`Asia/Shanghai`)清零**:联通为 27 号 0 点(业务确认,见 `docs/engineering/套餐流量计算逻辑核对.md`;与老卡管 `DateUtil.getLtPeriod` 一致),其余为 1 号 0 点。周期边界直接复用现有 `carrierthreshold.PeriodStart`(上海时区重置日 0 点),下文"今天日号""重置日当天"都按北京时间的日期判断。**查询失败、为空或不在 1~28 时,按卡的运营商类型兜底:`tb_iot_card.carrier_type = CUCC`(联通)取 27,其余取 1**,并记日志。不能沿用现有 `CarrierStore.GetDataResetDay` 一律返回 1 的做法(`carrier_store.go:103-104`)——联通卡被错当成 1 号重置会让自然月换算出错、累计值被抬高,按取最大值的规则会多扣且收不回。老路径继续用原函数,不改。
@@ -132,55 +164,73 @@
每次观测(轮询、人工刷新、系列观测三个入口)读到 `r`:
1. **无效读数和 0 不写**:查询失败、`used` 为空 / null / 无法解析 / `< 0`、或 `r = 0`,一律不写快照、不发增量。(老卡管普联通对 0 是"当天已有行不覆盖、当天第一条沿用昨天的值",移动是写一条 0;本方案直接不写,因为计算只看最大值,效果相同。)`FlowUsageResp` 增加"读数是否有效"标记,只给新代码用。
1. **无效读数和 0 不写**:查询失败、`used` 为空 / null / 无法解析 / `< 0`、或 `r = 0`,一律不写快照、不发增量。(老卡管普联通对 0 是"当天已有行不覆盖、当天第一条沿用昨天的值",移动是写一条 0;本方案直接不写,因为计算只看最大值,效果相同。)Gateway 的 `FlexFloat` 已把空、null、无法解析都转成 0(`internal/gateway/models.go:55-78`),与读数 0 走同一处理,**所以不需要新增"读数是否有效"标记**,Gateway 与三个观测入口都不用改。
2. **换算成自然月累计 `m`**:
- `R = 1`:`m = r`;
- `R > 1`,今天日号 `≥ R`:`m = r + max(本月 1 号~R−1 号的 month_used_mb)`;
- `R > 1`,今天日号 `< R`:`m = r − (上月最大 month_used_mb − 上月 1 号~R−1 号的最大 month_used_mb)`,< 0 按 0。
3. **重置日防旧值(完全照搬老卡管,比较基准也一致)**:只在重置日当天判断,原始读数照存 `vendor_used_mb`:
- `R = 1`:`r > 上月最大 month_used_mb × 0.9` 时按 `r = 0` 计算 `m`(同 `YidongDayFlowService.java:123`;上月没有快照时比较值为 0,任何正读数当天都按 0 记,次日如实写入,与老卡管一致);
- `R > 1`:本月 1 号~R−1 号的最大 `month_used_mb` 大于 0,且 `r` 超过它的 0.9 倍时,按 `r = 0` 计算 `m`(同 `PtLiantongDayFlowService.java:126`:`appendFlow > 0 && flow > appendFlow × 1.9`);本月前 R−1 天没有快照时不判断;
- `R = 1`:`r > 上月最大 month_used_mb × 0.9` 时按 `r = 0` 计算 `m`(同 `YidongDayFlowService.saveYidongDayFlow`;上月没有快照时比较值为 0,任何正读数当天都按 0 记,次日如实写入,与老卡管一致);
- `R > 1`:本月 1 号~R−1 号的最大 `month_used_mb` 大于 0,且 `r` 超过它的 0.9 倍时,按 `r = 0` 计算 `m`(同 `PtLiantongDayFlowService.saveLiantongDayFlow`:`appendFlow > 0 && flow > appendFlow × 1.9`);本月前 R−1 天没有快照时不判断;
- 触发时置 `stale_guarded = true`;第二天起不再判断,读数照常写入。
4. **同一天合并**:当天已有行时,`month_used_mb = max(旧值, m)`。
4. **同一天合并**:当天已有行时,`month_used_mb = max(旧值, m)`。这一步是改进,不是照搬:老卡管普联通同一天后写覆盖、可以变小,但新方案的增量一旦发出就收不回,所以快照只增不减。
5. **写法**:一律用 `INSERT ... ON CONFLICT (iot_card_id, flow_date) DO UPDATE SET month_used_mb = GREATEST(tb_card_flow_day.month_used_mb, EXCLUDED.month_used_mb), ...`,不先查后写。已切换卡在事务 1 内写,卡行已被锁(`traffic.go:40`),天然串行;未切换卡在事务 1 提交后写,没有卡锁,靠这条原子语句防止轮询与人工刷新并发写丢值。
### 4.4 今天的快照用量(照搬 3.4)
### 4.4 每天的快照用量(照搬 3.4,用于日流量展示)
```
今天用量 U(卡, 今天) = max(今天的 month_used_mb) − max(本月 1 号 ~ 昨天的 month_used_mb)
< 0 按 0;今天是本月第一条快照时减数为 0
U(卡, 某天) = max(那天的 month_used_mb) − max(本月 1 号 ~ 前一天的 month_used_mb)
< 0 按 0;那天是本月第一条快照时减数为 0
```
- 中间有几天没采到,那几天的用量自然并入今天,不会丢。
- 中间有几天没采到,那几天的用量自然并入采到的那天,不会丢。
- 每天的日流量就是 `U(卡, 那天)`,直接替代 Redis 日流量缓冲、02:00 落盘和 `tb_card_daily_usage`。
- **算增量不用这个公式**,用 4.5 的按月对账。两者按月合计相同,但按天对账遇到乱序观测会重复扣(见 4.5 说明)。
### 4.5 算增量并发出(替代 `r − 基线`)
在观测事务(事务 1,现有 `ApplyTraffic` 所在事务)里,对**已切换的卡**:
在观测事务(事务 1,现有 `ApplyTraffic` 所在事务)里,对**已切换且已有切换水位的卡**(没有水位的走 4.6):
1. `SELECT ... FOR UPDATE` 锁住该卡今天的快照行(不存在就插入);
2. 按 4.3 更新快照;
3. 增量 `Δ = U(卡, 今天) − emitted_mb`;
4. `Δ > 0` 且**该卡有可扣的生效套餐**(与 `resolveActivePackages` 同口径:卡自身有**生效中(status=1)**的套餐,或卡当前绑定的设备有生效中的套餐;**已用完(2)、待生效(0)不算**;只读查询,不加锁):写一条 V1 事件 `CardTrafficIncremented`,载荷与现在完全相同(只是增量值换了来源),同时 `emitted_mb += Δ`;
5. `Δ > 0` 但**没有生效套餐**:不发事件,只 `emitted_mb += Δ`。这部分用量不属于任何套餐,丢弃(日流量照常可见);
1. 卡行已在事务开头加锁(`traffic.go:40`),同卡观测天然串行,不再单独锁快照行;
2. 按 4.3 写今天的快照;
3. 按月对账:`M = 本月最大 month_used_mb`(写入后),`S = 本月各天 settled_mb 合计`,增量 `Δ = M − S`;
4. `Δ > 0` 且**该卡有可扣的生效套餐**(与 `resolveActivePackages` 同口径:卡自身有**生效中(status=1)**的套餐,或卡当前绑定的设备有生效中的套餐;**已用完(2)、待生效(0)不算**;只读查询,不加锁):写一条 V1 事件 `CardTrafficIncremented`,载荷与现在完全相同(只是增量值换了来源),同时今天的 `settled_mb += Δ`;
5. `Δ > 0` 但**没有生效套餐**:不发事件,只把今天的 `settled_mb += Δ`。这部分用量不属于任何套餐,丢弃(日流量照常可见);
6. `Δ ≤ 0`:不发事件。
7. **通道阈值判定**:本次读数有效(`r > 0`)且本次读数未被防旧值按 0 处理时,用 `r` 调用 `EvaluateInTx`,不再看老路径是否接受读数(见下方说明)。
说明:
- **增量与发出标记在同一事务里**,和 Outbox 事件一起提交或一起回滚,不会重复发、也不会漏发。同卡并发观测由快照行锁串行。
- **为什么按月对账**:若按天对账(`今天用量 − 今天已发出`),一次迟到的观测改写了昨天的快照,昨天会补发一笔,而今天的增量已按旧的"此前最大值"发过,两笔重叠的部分会扣两次(跨零点时轮询与人工刷新并发即可出现)。按月对账只看"本月最大值"和"本月已结算合计",与观测先后无关。
- **跨月**:新月份 `S = 0`,`M` 只取新月份的快照,第一条快照的增量就是本月累计。重置日为 1 号的卡,"上月最后一次采到 → 清零"之间的用量拿不到(见第 9 节"旧周期尾部")。
- **增量与已结算标记在同一事务里**,和 Outbox 事件一起提交或一起回滚,不会重复发、也不会漏发。
- **消费者完全不改**:现有消费者照旧调用 `DeductDataUsage` 扣减(先加油包、后主套餐、超额记最后一个),写流量详单,调用 `EvaluateAndAct` 判断停复机,回执表照旧保证幂等。
- **没有生效套餐时不发事件**:现有消费者遇到"没有可用套餐"会报错重试(第 2 节第 6 条),所以不能把这部分增量发出去。只推进 `emitted_mb`,**激活之前的用量永远不会扣到新套餐上**,二次销售天然正确,而且比老卡管按天算更精确:激活当天、激活之前的那部分也不会算给新用户。
- 是否有生效套餐只在事务 1 里只读判断一次,不改套餐侧代码;判断出错时按"有套餐"处理并发事件(保持现有行为,不丢量)。
- 卡表三列(`last_gateway_reading_mb` 等)和通道阈值判定:老 `ApplyTraffic` 照常计算、照常写,只是它算出的增量对已切换的卡不再使用。`last_gateway_reading_mb` 是观测的 CAS 条件,新代码只读不写。
- **已切换的卡不再发老增量的 V1**:老 `ApplyTraffic` 算出的 `decision.IncrementMB` 对已切换的卡丢弃,现在第 73-76 行的发事件逻辑只在未切换分支执行;每次观测只会发一条 V1,Outbox 量不增加。
- **没有生效套餐时不发事件**:现有消费者遇到"没有可用套餐"会报错重试(第 2 节第 6 条),所以不能把这部分增量发出去。只记为已结算:**激活之前被采到的用量不会扣到新套餐上**。精度到观测粒度:激活后第一次观测里,"上次观测到激活"这一段仍会算给新套餐(老卡管按天取值,激活当天的全部用量都算给新套餐)。
- 是否有生效套餐只在事务 1 里只读判断一次,不改套餐侧代码;判断出错时按"有套餐"处理并发事件(保持现有行为,不丢量)。事务 1 看到有生效套餐、但消费者扣减前套餐刚好被前一条事件扣完的情况,仍会报错重试,见第 9 节。
- 卡表三列(`last_gateway_reading_mb` 等):老 `ApplyTraffic` 照常计算、照常写,只是它算出的增量对已切换的卡不再使用。`last_gateway_reading_mb` 是观测的 CAS 条件,新代码只读不写。
- **卡表本月用量**:卡表 `current_month_usage_mb` 被资产详情(`service/asset/service.go:482`)、卡列表(`service/iot_card/service.go:501`)、设备导出(`exporter/device_scene.go:423`)直接读取。冻结卡按老逻辑这一列一直是 0,所以**已切换的卡在事务 1 里用本月最大快照 `M` 覆盖这一列**(写在同一条 `Updates` 里,老逻辑算出的值对已切换卡不用),三处读取都不用改。这一列只用于展示;补数据判定冻结卡时,已切换卡看水位、不看这一列(5.5),不受影响。跨月时老逻辑照常把上月值转入 `last_month_total_mb`,已切换卡转入的就是上月快照的最大值。
- **通道阈值**:老路径只在读数被接受时判定(`traffic.go:84`),冻结卡读数一直被拒,所以通道达量停机永远不触发;不改的话,全量切换后依然如此,只能等卡在重置窗口内被采到、基线自行重置。`EvaluateInTx` 只做"达量即写停机锁",没有"读数低就复机",所以放开判定不会误复机;排除被防旧值处理的读数,是为了防止重置日读到上周期旧总量、在新周期误停机;同一天之后读到的真实读数照常判定。此改动只在已切换分支内。
- **已切换的卡不再发老增量的 V1**:老 `ApplyTraffic` 算出的 `decision.IncrementMB` 对已切换的卡丢弃(切换那一次除外,见 4.6),现在第 73-76 行的发事件逻辑只在未切换分支执行;每次观测只会发一条 V1,Outbox 量不增加。
- 只有已用完或待生效套餐的卡,增量同样丢弃。这和现状一致:现状下这类增量会因"没有可用套餐"扣减失败并积压(第 2 节第 6 条),并不会扣到已用完的套餐上。
**未切换的卡**:事务 1 一切照旧(老增量、老 V1);快照在事务 1 提交后单独写,失败只告警(见 4.7),用于切换前积累数据和比对。
**未切换的卡**:事务 1 一切照旧(老增量、老 V1、老通道阈值条件);快照在事务 1 提交后按 4.3 第 5 步原子写入,失败只告警(见 4.7),用于切换前积累数据和比对;`settled_mb` 不动。
### 4.6 切换当天的衔接
### 4.6 切换衔接:切换水位
卡切换后的第一次观测,如果今天的快照行已经存在(切换前已经写过),先把 `emitted_mb` 初始化为"写入本次读数之前的 `U(卡, 今天)`",再按 4.5 计算。这样今天切换前已经由老路径扣过的部分不会再扣一次,只发本次观测带来的增长。
卡已在新路径名单中、但 `tb_card_flow_switch` 里还没有它的行时,这次观测是它的**第一次切换观测**,在事务 1 内:
限制:同一张卡**退回老路径后当天不要再切回**。当天快照上的 `emitted_mb` 已经不是 0,再切回时不会重新初始化,退回期间老路径扣过的部分会被再扣一次。
1. 按 4.3 写今天的快照,得到 `M`(本月最大)与 `S`(本月已结算合计,正常为 0);
2. 本次增量**照用老路径的结果** `decision.IncrementMB`:`> 0` 且有生效套餐时发 V1;无生效套餐时不发(与 4.5 一致,老路径这时会发);
3. 今天的 `settled_mb += max(M − S, 0)`,即把本月到此刻为止全部记为已结算;
4. 写入切换水位行(4.2),记下 `M`、`r`、卡表基线、卡表本月计入、老路径是否接受读数、老增量;
5. 本次的通道阈值仍按老路径条件判定,从下一次观测起按 4.5 第 7 步;卡表本月用量从本次起按 4.5 用 `M` 覆盖。
效果:
- 切换这一次的增量与老路径相同,之后只发切换之后的增长,**不依赖切换前本月是否已有快照**,阶段 1 不需要覆盖整月。
- 冻结卡:老路径本次增量为 0,切换前少扣的部分不在这里处理,由 5.5 按水位补。
- 退回老路径时删除水位行(5.2);再切回会重新走一次本节,退回期间老路径扣过的部分不会重扣。
- 写水位与快照、Outbox 在同一事务,失败随事务 1 回滚。
### 4.7 失败处理
@@ -188,7 +238,7 @@
|---|---|
| Gateway 查询失败 / 无效读数 / 读数为 0 | 不写快照、不发增量 |
| 运营商重置日查不到或非法 | 按运营商类型兜底:联通(`CUCC`)27 号 0 点,其余 1 号 0 点,北京时间;记日志。老路径照旧 |
| 已切换卡:快照写入失败 | 随事务 1 回滚,观测重入队(与现在观测失败的处理一致) |
| 已切换卡:快照、已结算标记或切换水位写入失败 | 随事务 1 回滚,观测重入队(与现在观测失败的处理一致) |
| 未切换卡:快照写入失败 | 只告警,不影响观测;总开关可整体停写 |
| 某天没采到 | 用量并入下一个有快照的日子 |
@@ -198,14 +248,17 @@
| # | 接触点 | 设计 | 结论 |
|---|---|---|---|
| 1 | 写快照(阶段 1 起全部卡都写) | 非灰度卡在观测事务**提交之后**单独写,失败、超时只告警,不让观测失败、不重入队;总开关 `JUNHONG_TRAFFIC_FLOW_SNAPSHOT_ENABLED` 关掉即停写,不用发版 | 不影响扣费与观测;数据库多一次写入(现约 2,400 次/小时),上线后观察轮询耗时,异常即关开关 |
| 1 | 写快照(阶段 1 起全部卡都写) | 非灰度卡在观测事务**提交之后**用一条 `INSERT ... ON CONFLICT DO UPDATE` 原子写入(4.3 第 5 步),失败、超时只告警,不让观测失败、不重入队;总开关 `JUNHONG_TRAFFIC_FLOW_SNAPSHOT_ENABLED` 关掉即停写,不用发版 | 不影响扣费与观测;数据库多一次写入(现约 2,400 次/小时),上线后观察轮询耗时,异常即关开关 |
| 2 | 判断走新路径还是老路径 | 观测事务内用启动时载入内存的卡集合判断,不查库;判断出错一律走老路径 | 不影响 |
| 3 | 增量计算 | 非灰度卡仍用"读数 − 基线",仍发 V1 事件,代码不改 | 不影响 |
| 4 | 无套餐时不发事件 | 只对灰度卡生效 | 不影响 |
| 5 | Gateway 返回加"读数是否有效"标记 | 只给新代码用,老路径照旧(无效仍按 0 处理) | 不影响 |
| 5 | Gateway 与三个观测入口 | 不改(不新增"读数是否有效"标记,见 4.3 第 1 步) | 不影响 |
| 6 | 日流量查询 | 非灰度卡仍读老来源 | 不影响 |
| 7 | 数据库迁移 | 只新建表,不改现有表 | 不影响 |
| 8 | 设备部分卡切换 | 不允许:设备整台切(5.2) | 不影响 |
| 9 | 通道阈值判定条件 | 只在已切换分支改为"读数有效且本次未被防旧值处理";未切换分支仍是"老路径接受读数" | 不影响 |
| 10 | 切换水位表 | 只有已切换卡读写 | 不影响 |
| 11 | 卡表 `current_month_usage_mb` | 只对已切换卡用快照覆盖;未切换卡仍由老逻辑写 | 不影响 |
**唯一无法靠设计消除的风险**:`ApplyTraffic` 是所有卡共用的代码,改动出错可能波及全部卡。约束:新逻辑只放在"判定为灰度卡"的分支里,老分支一行不改;评审重点核对这一处;测试环境回归"非灰度卡观测一次"的结果与改动前完全一致。
@@ -230,23 +283,25 @@
| 阶段 | 内容 | 对业务影响 |
|---|---|---|
| 1 写快照 | 建 `tb_card_flow_day`,全部卡在观测事务提交后写快照 | 无,只写不用 |
| 2 灰度切换 | 名单内的卡按 4.5 算增量;其余照旧 | 只影响名单内的卡 |
| 2 灰度切换 | 名单内的卡第一次观测按 4.6 写切换水位,之后按 4.5 算增量;其余照旧 | 只影响名单内的卡 |
| 3 全量 | 全部卡切换;之后下线 Redis 日流量缓冲与 02:00 落盘 | — |
阶段 1 至少跑满一个完整自然日再进入阶段 2;跨月或跨运营商重置日时,最好已有上个周期的快照(防旧值要用)。
阶段 1 至少跑满一个完整自然日再进入阶段 2。切换衔接靠切换水位(4.6),**不要求阶段 1 覆盖整月**。但防旧值和联通换算要用上个周期的快照:重置日为 1 号的卡,最好上月已有快照;联通卡最好上月 1~26 号已有快照,否则阶段 1 第一个月的联通快照会把上月 27 号以后的用量算进本月(只影响日流量展示的分摊,不影响按月对账的增量)。
### 5.2 切换方式
- 环境变量 `JUNHONG_TRAFFIC_CALC_MODE`:`off`(默认)/ `gray` / `full`;`JUNHONG_TRAFFIC_CALC_CARDS`:灰度名单,写扣减载体:`iot_card:103154`(单卡套餐的卡)或 `device:15276`(设备,展开为当前绑定的全部卡);`JUNHONG_TRAFFIC_FLOW_SNAPSHOT_ENABLED`:快照写入总开关,默认开。改完重启 API 和全部 Worker(见下方配置位置)。
- 判定在事务 1 里做,用启动时展开的内存集合,不查库;判定出错一律按老路径。
- **切换、扩批、退回都要先停掉 API 和全部 Worker,再一起启动,不逐个重启**。原因:逐个重启时会有新旧进程并存的窗口,同一张卡如果先被新进程写了水位、再被老进程观测(多为人工刷新),老进程发的那笔增量不计入已结算合计,下次新路径观测会再扣一次。窗口只有几分钟,概率很低,但停后再启即可完全避免。
- **配置位置**:新增 `pkg/config/config.go` 的 `TrafficCalc` 配置段与 `pkg/config/defaults/config.yaml` 的 `traffic_calc:` 默认值(`mode: off`、名单为空、快照开关开),沿用现有"嵌入默认配置 + `JUNHONG_` 环境变量覆盖"机制。生产上由维护者写入各运行目录的 `.env.prod`:**API(`/opt/junhong_cmp/api/.env.prod`,人工刷新走 API 进程)、调度 Worker(`/opt/junhong_cmp/worker/.env.prod`)和三个执行 Worker(`worker-1/2/3/.env.prod`)五处必须一致**,否则同一张卡在不同进程里会走不同路径。改完重启对应服务。
- **灰度单位是扣减载体,设备必须整台切**:名单里的设备,当前绑定的全部卡一起切换;名单里写 `iot_card:X` 但这张卡实际用的是设备套餐(与 `resolveActivePackages` 同口径),启动日志报错并跳过该项,不阻止 worker 启动。否则同一个设备套餐会同时收到新旧两种算法的增量,设备上没切换的卡也会被连带影响。
- **灰度期间灰度设备不换绑、不新绑卡**:名单在启动时展开成卡集合,之后新绑到灰度设备上的卡要重启后才会切换,这期间设备处于"部分切换"状态。确需换绑的,换绑后立即重启 API 和全部 Worker 重新展开;或先把该设备移出名单。
- **退回**:从名单删除或改回 `off` 并重启。老 `ApplyTraffic` 一直在维护卡表基线,退回后老路径接着算(冻结卡会回到冻结状态)。
- **退回**:从名单删除或改回 `off`,**删除这些卡在 `tb_card_flow_switch` 里的水位行**,再重启。老 `ApplyTraffic` 一直在维护卡表基线,退回后老路径接着算(冻结卡会回到冻结状态)。水位行必须删:否则以后再切回时不会重新走 4.6,退回期间老路径扣过的增长会被新路径再扣一次。退回与再切回可以在同一天。
- 灰度设备换绑后重启:新绑上的卡第一次走新路径时按 4.6 写水位,不会重复扣;换绑前后这台设备仍短暂处于"部分切换",所以上一条"不换绑"的约束保留。
### 5.3 冻结卡与历史误差
- 切换后冻结卡(如 103154)立即恢复正常计量:增量只看快照,不看卡表基线。
- 切换后冻结卡(如 103154)立即恢复正常计量:切换那一次老路径增量为 0(4.6),之后增量只看快照,不看卡表基线;通道阈值判定同时恢复(4.5 第 7 步)。
- **切换前少扣的部分放到后期补**,用独立脚本执行,见 5.5。
- **上个周期及更早少扣的补不回来**:运营商读数已经清零,没有任何记录;期间套餐已经重置的,那个计费周期也已结束。
- **第一批灰度:103154 所在设备 15276**(联通 103154、移动 99426、电信 106882,三张卡本月计入都为 0)。核对无误后再放其他冻结卡、电信读数常为 0 的卡、二次销售的卡、联通卡、带加油包的载体。
@@ -281,15 +336,16 @@
```
- 每张卡按自己运营商的重置日确定周期起点,设备下混着不同运营商的卡也能用;不需要按套餐拆分日记录。
- `r` 在执行时现场查询 Gateway,限速执行;查询失败、无效、读数为 0 的跳过,进清单。
- **未切换的卡**:`r` 在执行时现场查询 Gateway,限速执行;查询失败、无效、读数为 0 的跳过,进清单。
- **已切换的卡**:`r` 取切换水位的 `vendor_used_mb`(切换时的运营商读数),不现场查。切换之后的用量已由新路径计算,用水位截断,不会重复。切换时刻不在执行时所在运营商周期内的,进清单。
**判定为冻结卡(全部满足才自动补)**:
| 条件 | 原因 |
|---|---|
| 卡(或其所在设备)有**生效中(status=1)**的套餐(与 4.5 同口径) | 只有已用完或待生效套餐时,`DeductDataUsage` 找不到可扣套餐会报错,补扣事件会积压(第 2 节第 6 条);这类卡进清单 |
| 最近两天在被采集,本月计入(`current_month_usage_mb`)为 0 | 本周期没有任何读数被老路径接受 |
| 现场读数 `r` < 卡表基线 `last_gateway_reading_mb` | 基线属于上个周期或被异常值抬高,读数一直被拒 |
| 未切换卡:最近两天在被采集,本月计入(`current_month_usage_mb`)为 0。已切换卡:水位的 `old_reading_accepted = false` 且 `old_month_usage_mb = 0` | 本周期没有任何读数被老路径接受。已切换卡切换后卡表仍由老逻辑维护、本月计入可能一直是 0,所以只能看水位,不能看卡表 |
| `r` < 基线(未切换卡用现场读数与卡表 `last_gateway_reading_mb`;已切换卡用水位的 `vendor_used_mb` 与 `old_baseline_mb`) | 基线属于上个周期或被异常值抬高,读数一直被拒 |
| 卡所在扣减载体的套餐计费起点(激活或最近一次重置)**早于本运营商周期起点** | 否则 `r` 里有一部分属于已经结束的上个计费周期,不该补到新周期 |
| 本周期内没有回收换代、换绑 | 读数和套餐对不上;也排除了二次销售 |
@@ -299,7 +355,7 @@
1. **预演(只读)**:输出每张卡的卡号、运营商、扣减载体、`r`、基线、补扣量、补扣后是否超额;汇总补扣总量、预计停机数、清单与原因。
2. 确认后正式执行:每张卡写一条一次性的 V1 事件,事件 ID 由卡、运营商周期、批次组成,重复执行同一批次不重复写;由现有消费者调用 `DeductDataUsage` 扣减,超额照常停机,补扣量照常写入 `daily_record`。
3. 已切换到新方案的卡,切换之后的用量由新方案计算,补扣只补切换之前那段;脚本按"切换时刻已知的读数"截断,避免与新方案重复。
3. 已切换到新方案的卡,切换之后的用量由新方案计算,补扣只补切换之前那段:补扣量取水位的 `vendor_used_mb`,避免与新方案重复。
4. 补扣写审计,注明"冻结卡补扣"与补扣量。
5. 现有消费者会把每条 V1 事件的增量累加进当天的 Redis 日流量缓冲,所以**未切换卡的日流量会在补扣当天出现一次突增**(已切换卡的日流量读快照,不受影响)。执行前告知运营,或在预演清单中标出。
@@ -307,12 +363,11 @@
| 位置 | 改动 |
|---|---|
| `migrations/` | 新建 `tb_card_flow_day` |
| `internal/gateway/models.go` | `FlowUsageResp` 增加"读数是否有效"标记(老路径行为不变) |
| 新增 `internal/domain/cardflow/` | 纯函数:自然月换算、重置日防旧值、今天用量;表驱动单测 |
| 新增 `internal/store/postgres/card_flow_day_store.go` | 快照读写(含行锁) |
| `internal/application/cardobservation/traffic.go` | 已切换的卡:增量改为 4.5 的算法,V1 事件载荷不变,按卡的 `carrier_id` 严格查重置日;未切换的卡:老逻辑不变,提交后写快照 |
| 三个观测入口(`polling_carddata_handler.go`、`service/iot_card/service.go`、`series_runner.go`) | 只把"读数是否有效"传进 `TrafficObservation`(新增字段 `ReadingInvalid`,默认值表示有效,老调用方不受影响);快照的写入都在 `ApplyTrafficObservation` 里完成:已切换的卡在事务 1 内写,未切换的卡在事务 1 提交后写 |
| `migrations/` | 新建 `tb_card_flow_day`、`tb_card_flow_switch` |
| 新增 `internal/domain/cardflow/` | 纯函数:自然月换算、重置日防旧值、每天用量、按月对账增量;表驱动单测 |
| 新增 `internal/store/postgres/card_flow_day_store.go` | 快照原子写入(`ON CONFLICT DO UPDATE`)、本月最大值与已结算合计查询 |
| 新增 `internal/store/postgres/card_flow_switch_store.go` | 切换水位读写 |
| `internal/application/cardobservation/traffic.go` | 已切换的卡:增量改为 4.5 的按月对账,第一次切换观测按 4.6 写水位,通道阈值按 4.5 第 7 步判定,卡表 `current_month_usage_mb` 用本月最大快照覆盖,V1 事件载荷不变,按卡的 `carrier_id` 严格查重置日;未切换的卡:老逻辑不变,提交后写快照。快照写入都在 `ApplyTrafficObservation` 里完成,三个观测入口与 Gateway 不改 |
| 日流量查询 | 已切换的卡读快照 |
| `pkg/config` | 三个环境变量(模式、灰度名单、快照写入总开关) |
| 后期:补数据脚本(`cmd/`) | 预演(只读)+ 正式执行(写 V1 事件),见 5.5 |
@@ -329,7 +384,7 @@
| 重置日 `R = 15`(任意非 1 号) | 换算规则同上,以 15 号为界,结果连续 |
| 重置日 `R = 1` | 不换算,`month_used_mb` = 读数 |
| 运营商的 `data_reset_day` 被修改 | 从修改后的下一次写入起按新重置日换算 |
| `R > 1` 时 1 号~R−1 号换算 | 减去上月 R 号~月底的用量,按"1 号~R−1 号"取值,不少一天 |
| `R > 1` 时 1 号~R−1 号换算 | 减去上月 R 号~月底的用量(= 上月最大 − 上月 1~R−1 号最大,含 R−1 号) |
| 重置日当天读到上周期旧值(> 90%) | 当天按 0 记,`stale_guarded = true`;之后读到真实低值正常累计 |
| 重置日次日起仍读到旧值 | 照常累加(与老卡管一致) |
| 重置日当天真实用量超过上周期 90% | 当天按 0 记;次日如实写入,取最大值把这部分带回来,只晚一天,不丢 |
@@ -340,20 +395,30 @@
| `R = 1` 重置日当天读数 > 上月最大快照 × 0.9 | 当天按 0 记 |
| `R = 27` 重置日当天读数 > 本月 1~26 号最大快照 × 0.9 | 当天按 0 记 |
| 中间 3 天没采到 | 3 天的用量并入采到的那天 |
| 跨自然月 | 本月第一条快照减数为 0,从本月累计开始算 |
| 跨自然月 | 本月第一条快照减数为 0,从本月累计开始算;按月对账时新月份已结算合计为 0 |
| 按月对账:本月最大 500、已结算合计 450 | 增量 50 |
| 乱序:今天已发出后,一次迟到观测抬高了昨天的快照 | 按月对账的增量 = 本月最大 − 已结算合计,不会把重叠部分扣两次 |
**应用层**
| 场景 | 期望 |
|---|---|
| **卡 103154 类:卡表基线 72535,运营商读数 48808** | 切换后增量按快照正常发出,套餐恢复增长 |
| **二次销售:上一个用户本月用到 3000;新用户 15 号 14:00 激活;激活前当天又读到 3020,激活后读到 3050** | 3020 及之前的增量在无套餐时只推进 `emitted_mb`、不发事件;新套餐只扣 30 |
| **二次销售:上一个用户本月用到 3000;新用户 15 号 14:00 激活;激活前当天又读到 3020,激活后读到 3050** | 3020 及之前的增量在无套餐时只记入 `settled_mb`、不发事件;新套餐只扣 30 |
| 卡只有已用完或待生效的套餐 | 不发事件,增量丢弃 |
| 灰度卡观测一次 | 只发一条 V1(新算法增量),不再发老增量的 V1 |
| 无套餐期间连续观测,之后激活 | Outbox 里没有该卡积压的 V1 事件;激活后只扣激活之后的增长 |
| 同一次观测重复处理 / 同卡并发观测 | 行锁串行,`emitted_mb` 保证每段增长只发一次 |
| 事务 1 回滚 | 快照、`emitted_mb`、Outbox 一起回滚,不重不漏 |
| 切换当天切换前老路径已扣过一部分 | 按 4.6 初始化,不重复扣 |
| 同一次观测重复处理 / 同卡并发观测 | 卡行锁串行,按月对账保证每段增长只结算一次 |
| 事务 1 回滚 | 快照、`settled_mb`、切换水位、Outbox 一起回滚,不重不漏 |
| 第一次切换观测:正常卡,本月已有快照 | 只发老路径这次的增量;写水位;本月已结算合计 = 本月最大 |
| 第一次切换观测:正常卡,本月还没有快照 | 同上,不会把整月累计当增量发出 |
| 第一次切换观测:冻结卡(老路径拒绝读数) | 不发增量;水位记 `old_reading_accepted = false`、`vendor_used_mb = r`;下一次观测起按快照正常发 |
| 退回老路径(删水位)后当天再切回 | 重新按 4.6 写水位,退回期间老路径扣过的部分不重扣 |
| 冻结卡切换后读数超过通道阈值 | 写通道停机锁(老路径下不会触发) |
| 冻结卡切换后 | 资产详情、卡列表、设备导出的本月用量 = 本月最大快照,不再是 0 |
| 重置日当天先触发防旧值、之后读到真实低值 | 真实读数照常做通道阈值判定 |
| 重置日当天读到上周期旧总量并触发防旧值 | 不做通道阈值判定,不误停机 |
| 未切换的卡并发写快照(轮询与人工刷新同时) | 原子写入,`month_used_mb` 取较大值,不丢 |
| 名单写 `device:15276` | 设备当前绑定的 3 张卡全部切换 |
| 联通卡观测时运营商表查询失败(入口传进来的 `ResetDay` 已被兜底成 1) | 新逻辑不用入口的值,自行严格查询,失败按 27 号处理 |
| 名单写 `iot_card:X`,但该卡实际用设备套餐 | 启动日志报错并跳过该项,worker 正常启动,该卡仍走老路径 |
@@ -375,6 +440,8 @@
| 冻结卡只有已用完的套餐 | 不补,进清单 |
| 少扣为负(多扣) | 不退,进清单 |
| 同一批次重复执行 | 事件 ID 相同,不重复写 |
| 已切换的冻结卡 | 补扣量取水位的 `vendor_used_mb`,不现场查;切换之后的用量不重复补 |
| 已切换的正常卡(水位 `old_reading_accepted = true`) | 不补 |
| 套餐计费起点在本周期内的冻结卡 | 不自动补,进清单并附按时间比例的估算值 |
**测试环境验证**:103154 同类卡、三卡共用设备、二次销售卡、重置日为 27 号与 1 号的运营商各跨一次重置日、另构造一个非常规重置日(如 15 号)各一组,逐日核对快照、增量与套餐已用量。
@@ -384,14 +451,16 @@
| # | 任务 | 人天 |
|---|---|:-:|
| 1 | 迁移与模型 | 0.25 |
| 2 | 领域层:自然月换算、防旧值、今天用量 + 单测 | 1.5 |
| 3 | 快照存储(含行锁)+ 三个观测入口写快照 + 无效读数标记 | 1.0 |
| 4 | `ApplyTraffic` 已切换卡的增量替换与切换当天衔接 | 1.0 |
| 2 | 领域层:自然月换算、防旧值、每天用量、按月对账 + 单测 | 1.5 |
| 3 | 快照存储(原子写入)+ 切换水位存储 | 0.75 |
| 4 | `ApplyTraffic` 已切换卡:按月对账、切换水位、通道阈值条件 | 1.25 |
| 5 | 切换机制(模式、名单、启动展开) | 0.5 |
| 6 | 日流量读侧 | 0.5 |
| 7 | 集成测试与测试环境验证 | 1.5 |
| | 合计 | 约 6.25 |
与第 1 版相比:去掉"读数是否有效"标记和三个入口的改动,省下的工作量用于切换水位与通道阈值条件,合计不变。
后期补数据脚本(5.5:预演、条件判断、写 V1 事件、清单)另计约 1.5 人天。
不含灰度观察期:至少跨过一次运营商清零,约 30 天内。
@@ -401,11 +470,128 @@
| 限制 | 说明 |
|---|---|
| 切换前的误差 | 后期按 5.5 补,只能补执行时所在运营商周期内的部分;更早的补不回来;不满足条件的进人工清单 |
| 扣减前套餐刚好用完 | 事务 1 看到有生效套餐并发出事件,但消费者扣减前,同卡的前一条事件已把唯一的套餐扣到"已用完",这条事件扣减时报"没有可用套餐"并重试,之后可能扣到新激活的套餐上(第 2 节第 6 条的同一条路径)。需要同卡在很短时间内有两条事件,概率低。消费者不改(D13),灰度期监控"处理中"回执的新增数,确有发生再评估是否让消费者对已切换卡直接结束 |
| 二次销售精度 | 只精确到观测粒度:激活后第一次观测里"上次观测到激活"的用量会算给新套餐。要更精确需要激活时触发一次立即观测(改激活入口,本期不做) |
| 切换那一次沿用老路径增量 | 第一次切换观测发的是老路径算出的增量(4.6),如果这一次正好撞上老路径的问题(例如重置窗口读到 0 把基线置 0 后,又读到真实值整笔计入),会照样扣一次。每张卡最多一次 |
| 换绑后第一次观测 | 设备换上一张本月没有快照的卡,第一次观测(走 4.6)发出的是老路径的增量,与现状一致:老路径对从未采过的卡基线为 0,会把整张卡的周期读数算给设备套餐 |
| 上一代套餐仍为"生效中" | `DeductDataUsage` 只按状态选套餐、不看世代,回收换代后上一代套餐如果仍为生效中,新用户的增量可能扣到它上面。这是套餐侧问题,按 `docs/资产回收换代技术方案.md` 单独修复,本方案不改套餐侧 |
| 旧周期尾部 | 旧周期最后一次采集到清零之间的用量拿不到,老卡管同样如此 |
| 防旧值 | 完全照搬老卡管:阈值 0.9、只在重置日当天判断。运营商延迟清零超过一天时,次日的旧值会被累加(老卡管同样如此,多年未暴露问题) |
| 积压的无套餐回执 | 18,979 条卡在"处理中"的回执(含 4,141 条对应卡当前有生效套餐)由业务后续手动调整,不在本方案内;处理前不得重放 |
| 修改运营商重置日 | 月中修改 `data_reset_day` 会让当月换算前后不连续,可能产生一次跳变多扣。修改应安排在运营商实际重置日前后,并在修改后核对受影响卡的当日增量 |
| 采集不足 | 轮询吞吐问题(约 2.4k 次/小时)不在本方案范围;采不到只会晚扣,不会丢 |
| 采集不足 | 轮询吞吐问题(约 2.4k 次/小时,平均每卡约 1.5 天一次)不在本方案范围;采不到只会晚扣,不会丢。但每张卡在自己的重置日(1 号或联通 27 号),最后一次采集到清零之间的用量会丢("旧周期尾部"),采得越稀丢得越多;重置日当天没采到的卡也不会触发防旧值 |
| 扣错收不回 | 套餐侧仍是增量扣减,一旦扣错只能人工改已用量。快照可以完整复算每天应扣多少,作为核对依据 |
| 电信 | 老卡管的电信是直接按日期区间向运营商查询(`TelecomUtilNew.queryThisMonthDianxinDayFlow`);新卡管 Gateway 目前只返回当前周期累计,本期统一按快照处理,读数为 0 的电信卡仍扣不到。**后续安排 Gateway 支持按日期区间查询**:支持后,电信卡改为按区间查询的结果写快照,算增量的逻辑不变;同时可以用来补旧周期尾部、补更早周期的少扣数据 |
| 电信 | 老卡管的电信在采集时按"本月 1 号~今天"向运营商查询(`TelecomUtilNew.queryThisMonthDianxinDayFlow`),拿到自然月累计后同样写日快照、按快照计费;新卡管 Gateway 目前只返回当前周期累计,本期统一按快照处理,读数为 0 的电信卡仍扣不到。**后续安排 Gateway 支持按日期区间查询**:支持后,电信卡改为按区间查询的结果写快照,算增量的逻辑不变;同时可以用来补旧周期尾部、补更早周期的少扣数据 |
## 10. 停复机规则与实名管控
本章与流量计算相互独立,可以分开开发、分开上线;放在同一份方案里一起评审。
### 10.1 停复机原则(已定)
| | 条件 | 说明 |
|---|---|---|
| **停机** | 卡开机,且满足任一:① 没有套餐(套餐到期);② 流量用完;③ 卡所属通道开启了实名管控(`realname_control_enabled = 1`)且卡未实名 | 只有这三条。谁开的机不重要 |
| **复机** | 卡停机,且同时满足:① 有生效套餐;② 流量没用完;③ 卡已实名,**或**卡所属通道关闭了实名管控(`realname_control_enabled = 0`) | **不看停机原因**:外部停的、卡管自动停的、管理员手动停的、设备手动停的,满足条件都复机 |
| 不动作 | 卡或所在设备暂停轮询;卡持有本周期通道阈值锁(属于"流量用完",锁期内不复机) | 见 10.4 |
- **外部停机与卡管无关**:外部环境导致的停机,卡管判断该开机就复机,**除非这张卡(或所在设备)已被加入暂停轮询**。卡管不判断是谁停的,只按上面的条件决定开或停;外部反复停卡,卡管就按规则反复复机,不设次数限制。
- **实名默认不管控**:通道上开启"实名管控"后,未实名的卡会被停机,停机后实名了才复机;没开启的通道,实名不参与停复机。"实名管控"是通道上**新增的独立开关**(`realname_control_enabled`),只管停复机,**不复用通道现有的实名相关字段**(如实名链接类型 `realname_link_type`,它只决定实名链接怎么生成)。开启后,通道下所有卡都按"是否已实名"判断,**不再区分行业卡,也不再看卡或设备上的实名策略**(实名策略只用于购买流程)。
- `stop_reason` 仍照常记录(流量耗尽、无套餐、未实名、手动、运营商侧停机、通道阈值),**只用于展示和审计,不参与复机判断**。
### 10.2 现状与差异(代码核实)
| # | 现在的行为 | 与原则的差异 | 代码 |
|---|---|---|---|
| 1 | 自动复机只处理停机原因为流量耗尽、无套餐、未实名的卡 | 外部停机(记为 `carrier_stopped`)、手动停机(`manual`)、设备保护期停机(`protect_period`)满足条件也不复机,只能人工处理 | `stop_resume_service.go` 的 `isPollingStopReason`、`EvaluateAndAct` 停机分支、`resumeSingleCard`;`iot_card_store.go` 的 `ListByDeviceIDAndPollingStopReasons` |
| 2 | 非行业卡、未实名、卡或设备实名策略不是 `none` 时一律停机且不能复机 | 实名默认管控,且与行业卡、资产实名策略耦合;想放开只能逐张改实名策略 | `isRealnameOK`,被 `checkStopReasons`、`shouldResume`、`resumeDeviceCards`、`ManualStartCard` 共用;保护期一致性检查跳过未实名卡(`polling_protect_handler.go:77`) |
停机这一侧现在已经只看无套餐、流量耗尽、未实名,符合原则;停机和复机里的实名都改成按通道判断。
### 10.3 已定决策(2026-10-20)
| # | 事项 | 决定 |
|---|---|---|
| D14 | 停机条件 | 无套餐、流量用完、通道开启实名管控且卡未实名,三者任一 |
| D15 | 复机条件 | 有生效套餐、流量没用完、卡已实名或通道关闭了实名管控,三条同时满足即复机;**不看停机原因**(含外部停机、手动停机、设备手动停机) |
| D16 | 实名管控 | 通道(`tb_carrier`)新增独立开关 `realname_control_enabled`,不复用现有实名字段,**默认 0 = 不管控**;设为 1 时通道下所有卡按是否已实名判断(未实名停机、实名后才复机),不区分行业卡、不看资产实名策略。**本期只加通道开关,不迁移、不修改任何卡或设备数据**;卡级、设备级的实名管控以后需要时再加(同样用独立字段,默认继承通道) |
| D17 | 暂停轮询的卡 | 卡或所在设备暂停轮询时,卡管不自动停复机,人工停复机照常。风险停机、机卡分离停机、已销户等运营商异常状态**由人工判断并手动暂停轮询**,卡管不自动处理;现有"独立卡风险停机/已销户自动暂停轮询"的逻辑不改 |
### 10.4 设计
**统一判断**:新增一个"目标状态"判断,`EvaluateAndAct` 开机分支、停机分支、`resumeSingleCard`、`resumeDeviceCards`、`ResumeCardIfStopped` 都用它,不再各自判断。
- 开头先看暂停轮询:卡 `enable_polling = false`,或卡所在设备 `enable_polling = false`(与轮询范围 `ShouldEnqueue` 同口径),本次不动作。
- 卡开机:命中 10.1 的停机条件即停机,停机原因按命中的条件记录。设备维度扩散规则不变(无套餐、流量用完扩散到设备;未实名只停该卡)。
- 卡停机:满足 10.1 的复机条件即复机,**删除"只处理轮询原因停机"的限制**(`isPollingStopReason`)。持有通道阈值锁时不复机(现有 `channelThresholdBlocked`,不变)。
- 设备复机:`ListByDeviceIDAndPollingStopReasons` 改为取设备下全部停机卡,逐张按复机条件判断。
- 套餐判断沿用现有口径(`HasValidByCarrier`:生效中或待生效都算有套餐;流量用完沿用 `isTrafficExhausted`),本章不改套餐口径。
**实名管控(D16)**:
- 迁移:`tb_carrier` 增加 `realname_control_enabled smallint not null default 0`。
- `isRealnameOK`(停机判断、复机判断、设备复机、人工复机共用)改为:卡所属通道(`card.carrier_id`)`realname_control_enabled = 0` → 满足;= 1 → 卡 `real_name_status` 为已实名才满足。**删除行业卡放行和卡/设备实名策略判断**。查询通道失败时本次不动作并记日志(不按任何默认值停机或复机)。
- 保护期一致性检查:只在通道开启实名管控且卡未实名时跳过。
- 人工复机(`ManualStartCard`)同样经过 `isRealnameOK`,自动一致。
- 通道管理接口(`service/carrier` 创建、修改、详情)增加该字段,修改写审计。从 0 改为 1 时,管理端二次确认并提示该通道未实名的开机卡数(下一次评估会被停机);从 1 改为 0 后,该通道因未实名停机的卡会在下一次评估时复机。
- 卡和设备上的实名策略(`none` / `before_order` / `after_order`)、首次实名激活套餐,只用于购买与激活流程,不受影响。
**暂停轮询的卡(D17)**:
- 运营商异常状态(风险停机、机卡分离停机、已销户)不做自动处理,由人工判断后在管理端暂停轮询(现有接口)。现有独立卡风险停机/已销户自动暂停轮询的逻辑(`ShouldStopPollingForRisk`)保持不变。
- 暂停轮询后卡管不自动停复机(10.4 开头的检查),人工处理完再开启轮询,之后按 10.1 正常管理。
- 没有被暂停轮询的异常状态卡,卡管照常按 10.1 判断;复机请求被上游拒绝时按现有失败处理。
### 10.5 上线
新规则上线后会有一批卡集中动作,用一个总开关控制:
| 开关 | 默认 | 作用 |
|---|---|---|
| `JUNHONG_STOP_RESUME_UNIFIED_RULE_ENABLED` | `false` | 开启 10.1 的新规则(复机不看停机原因、实名按通道判断);关闭时与现在完全一致 |
| `tb_carrier.realname_control_enabled` | `0` | 按通道配置;只在总开关开启后生效 |
暂停轮询的卡不自动停复机(D17)也受总开关控制。
步骤:
1. 发版,总开关保持关闭。
2. **确认哪些通道需要实名管控**,在管理端设为 1。新规则默认不管控实名,没设置的通道,现在因未实名停机的卡在总开关打开后都会复机。
3. **打开总开关前预演(只读 SQL)**,统计下一次评估会集中动作的卡:
- 会被复机:停机、未暂停轮询、有套餐、流量没用完、(通道开启实名管控时已实名),按停机原因分组(`carrier_stopped`、`manual`、`protect_period`、`not_realname` 等)。
- 会被停机:开机、所属通道开启实名管控、未实名的卡(含行业卡,原来行业卡不停)。
4. 业务确认后打开总开关,先停掉 API 和全部 Worker 再一起启动(与 5.2 相同);观察停复机审计和上游失败数。
### 10.6 改动清单
| 位置 | 改动 |
|---|---|
| `migrations/` | `tb_carrier` 增加 `realname_control_enabled` |
| `internal/model/carrier.go`、`service/carrier`、相关 DTO | 实名管控字段的读写与审计 |
| `internal/service/iot_card/stop_resume_service.go` | 目标状态判断(10.4);复机不看停机原因;`isRealnameOK` 改为按通道判断;暂停轮询不动作;总开关 |
| `internal/store/postgres/iot_card_store.go` | 设备复机改为取设备下全部停机卡 |
| `internal/task/polling_protect_handler.go` | 只在通道开启实名管控且卡未实名时跳过 |
| `pkg/config` | 总开关 |
| **外部停机仍记 `carrier_stopped`、套餐口径、流量侧、购买与激活流程的实名策略** | **不改** |
### 10.7 测试用例
| 场景 | 期望 |
|---|---|
| 开机卡套餐到期 | 停机,原因 `no_package`(设备卡扩散到设备) |
| 开机卡流量用完 | 停机,原因 `traffic_exhausted` |
| 外部停机的卡,有套餐、流量没用完 | 自动复机 |
| 管理员手动停机的卡,有套餐、流量没用完 | 下一次评估自动复机 |
| 设备手动停机,保护期结束后 | 下一次评估按复机条件复机 |
| 停机卡没有套餐 | 保持停机;充值激活后复机 |
| 停机卡持有通道阈值锁 | 锁期内不复机 |
| 通道未开启实名管控(默认),未实名卡有套餐 | 不停机;已因未实名停机的卡复机;人工复机不因未实名被拒 |
| 通道开启实名管控,未实名的卡开着机、有套餐 | 停机,原因 `not_realname`,只停这张卡 |
| 通道开启实名管控,未实名的卡停着机、有套餐 | 不复机;实名后下一次评估复机;人工复机被拒 |
| 通道开启实名管控,未实名的行业卡 | 停机(不再区分行业卡) |
| 设备复机时设备下有一张未实名卡(通道开启实名管控) | 跳过这张卡(停机原因记为 `not_realname`),其他卡照常复机 |
| 通道开启实名管控,卡已实名、实名策略为 `before_order` | 按有套餐、流量判断,实名策略不参与 |
| 查询通道实名管控失败 | 本次不动作,记日志 |
| 设备下混有开启与未开启实名管控的通道的卡 | 各卡按自己通道判断实名;套餐与流量按设备维度 |
| 暂停轮询的卡(卡或所在设备暂停),充值激活触发评估 | 不停不复;人工停复机照常 |
| 总开关关闭 | 与现在完全一致 |

2
go.mod
View File

@@ -48,6 +48,7 @@ require (
github.com/go-playground/universal-translator v0.18.1 // indirect
github.com/go-sql-driver/mysql v1.8.1 // indirect
github.com/go-viper/mapstructure/v2 v2.4.0 // indirect
github.com/golang/snappy v0.0.1 // indirect
github.com/jackc/pgpassfile v1.0.0 // indirect
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 // indirect
github.com/jackc/puddle/v2 v2.2.2 // indirect
@@ -61,6 +62,7 @@ require (
github.com/mattn/go-isatty v0.0.20 // indirect
github.com/mattn/go-runewidth v0.0.16 // indirect
github.com/mohae/deepcopy v0.0.0-20170929034955-c48cc78d4826 // indirect
github.com/nsqio/go-nsq v1.1.0 // indirect
github.com/patrickmn/go-cache v2.1.0+incompatible // indirect
github.com/pelletier/go-toml/v2 v2.2.4 // indirect
github.com/philhofer/fwd v1.1.3-0.20240916144458-20a13a1f6b7c // indirect

4
go.sum
View File

@@ -106,6 +106,8 @@ github.com/golang-sql/civil v0.0.0-20220223132316-b832511892a9 h1:au07oEsX2xN0kt
github.com/golang-sql/civil v0.0.0-20220223132316-b832511892a9/go.mod h1:8vg3r2VgvsThLBIFL93Qb5yWzgyZWhEmBwUJWevAkK0=
github.com/golang-sql/sqlexp v0.1.0 h1:ZCD6MBpcuOVfGVqsEmY5/4FtYiKz6tSyUv9LPEDei6A=
github.com/golang-sql/sqlexp v0.1.0/go.mod h1:J4ad9Vo8ZCWQ2GMrC4UCQy1JpCbwU9m3EOqtpKwwwHI=
github.com/golang/snappy v0.0.1 h1:Qgr9rKW7uDUkrbSmQeiDsGa8SjGyCOGtuasMWwvp2P4=
github.com/golang/snappy v0.0.1/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q=
github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU=
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
@@ -175,6 +177,8 @@ github.com/mohae/deepcopy v0.0.0-20170929034955-c48cc78d4826 h1:RWengNIwukTxcDr9
github.com/mohae/deepcopy v0.0.0-20170929034955-c48cc78d4826/go.mod h1:TaXosZuwdSHYgviHp1DAtfrULt5eUgsSMsZf+YrPgl8=
github.com/morikuni/aec v1.0.0 h1:nP9CBfwrvYnBRgY6qfDQkygYDmYwOilePFkwzv4dU8A=
github.com/morikuni/aec v1.0.0/go.mod h1:BbKIizmSmc5MMPqRYbxO4ZU0S0+P200+tUnFx7PXmsc=
github.com/nsqio/go-nsq v1.1.0 h1:PQg+xxiUjA7V+TLdXw7nVrJ5Jbl3sN86EhGCQj4+FYE=
github.com/nsqio/go-nsq v1.1.0/go.mod h1:vKq36oyeVXgsS5Q8YEO7WghqidAVXQlcFxzQbQTuDEY=
github.com/opencontainers/go-digest v1.0.0 h1:apOUWs51W5PlhuyGyz9FCeeBIOUDA/6nW8Oi/yOhh5U=
github.com/opencontainers/go-digest v1.0.0/go.mod h1:0JzlMkj0TRzQZfJkVvzbP0HBR3IKzErnv2BNG4W4MAM=
github.com/opencontainers/image-spec v1.1.1 h1:y0fUlFfIZhPF1W537XOLg0/fcx6zcHCJwooC2xJA040=

View File

@@ -13,6 +13,7 @@ import (
"github.com/break/junhong_cmp_fiber/pkg/constants"
"github.com/break/junhong_cmp_fiber/pkg/errors"
"github.com/break/junhong_cmp_fiber/pkg/outboxid"
"go.uber.org/zap"
"gorm.io/gorm"
"gorm.io/gorm/clause"
)
@@ -73,6 +74,24 @@ type Service struct {
auditWriter StateAuditWriter
// channelThreshold 为可选的通道阈值判定能力;未注入时流量观测不做阈值判定。
channelThreshold ChannelThresholdEvaluator
// trafficCalc 判定卡是否走日快照计量算法;未注入时全部卡走卡表基线算法且不写快照。
trafficCalc *TrafficCalcPolicy
// logger 记录快照写入失败等不影响观测结果的告警;未注入时不输出。
logger *zap.Logger
}
// SetTrafficCalcPolicy 注入卡流量计量切换策略与告警日志。
func (s *Service) SetTrafficCalcPolicy(policy *TrafficCalcPolicy, logger *zap.Logger) {
s.trafficCalc = policy
s.logger = logger
}
// warnLogger 返回可用的告警日志器。
func (s *Service) warnLogger() *zap.Logger {
if s.logger == nil {
return zap.NewNop()
}
return s.logger
}
// NewService 创建卡实名观测应用服务。
@@ -272,3 +291,11 @@ func realnameChangedEventID(cardID uint, observationID string) string {
prefix := "card-realname:"
return outboxid.Stable(prefix, strconv.FormatUint(uint64(cardID), 10)+":"+observationID+":changed")
}
// TrafficSwitched 判定卡是否已切换到日快照计量(激活前流量结算只对已切换的卡执行)。
func (s *Service) TrafficSwitched(cardID uint) bool {
if s == nil {
return false
}
return s.trafficCalc.Switched(cardID)
}

View File

@@ -46,7 +46,7 @@ func (s *Service) ApplyNetworkObservation(ctx context.Context, observation domai
auditedCard = &card
nextDecision, decisionErr := domain.ApplyNetwork(domain.CardNetworkSnapshot{
CardID: card.ID, NetworkStatus: card.NetworkStatus, StopReason: card.StopReason,
IsStandalone: card.IsStandalone, EnablePolling: card.EnablePolling,
EnablePolling: card.EnablePolling,
}, observation)
if decisionErr != nil {
return decisionErr

View File

@@ -8,6 +8,7 @@ import (
"gorm.io/gorm"
"gorm.io/gorm/clause"
cardflow "github.com/break/junhong_cmp_fiber/internal/domain/cardflow"
domain "github.com/break/junhong_cmp_fiber/internal/domain/cardobservation"
carrierthresholddomain "github.com/break/junhong_cmp_fiber/internal/domain/carrierthreshold"
"github.com/break/junhong_cmp_fiber/internal/model"
@@ -53,6 +54,23 @@ func (s *Service) ApplyTrafficObservation(ctx context.Context, observation domai
return decisionErr
}
decision = nextDecision
// 已切换日快照算法的卡:增量、通道阈值与本月用量改由快照决定;读数为 0 或无效时不写快照、不产生增量。
switched := s.trafficCalc.Switched(card.ID)
var flow *switchedFlowOutcome
if switched && cardflow.ValidReading(observation.GatewayReadingMB) {
outcome, flowErr := s.applySwitchedFlow(ctx, tx, &card, observation, decision)
if flowErr != nil {
return flowErr
}
flow = outcome
}
if switched {
decision.IncrementMB = 0
if flow != nil {
decision.IncrementMB = flow.EmitMB
decision.CurrentMonthUsageMB = flow.MonthMaxMB
}
}
updates := map[string]any{
"last_data_check_at": observation.Metadata.ObservedAt, "last_sync_time": observation.Metadata.ObservedAt,
"current_month_start_date": decision.CurrentMonthStartDate, "last_month_total_mb": decision.LastMonthTotalMB,
@@ -71,6 +89,7 @@ func (s *Service) ApplyTrafficObservation(ctx context.Context, observation domai
return errors.New(errors.CodeConflict, "卡流量基线已被其他请求更新")
}
if decision.IncrementMB > 0 {
// 未切换卡沿用卡表基线算法的增量;已切换卡的增量已替换为按月对账结果,每次观测最多一条 V1 事件。
eventID := outboxid.Stable("card-traffic:", strconv.FormatUint(uint64(card.ID), 10)+":"+observation.Metadata.ObservationID+":incremented")
if err := s.eventWriter.AppendTraffic(ctx, tx, TrafficIncrementedEvent{
EventID: eventID, CardID: card.ID, IncrementMB: decision.IncrementMB,
@@ -81,12 +100,21 @@ func (s *Service) ApplyTrafficObservation(ctx context.Context, observation domai
return err
}
}
if decision.ReadingAccepted && s.channelThreshold != nil {
// 达量判定只使用本次已接受的网关累计读数,异常下降保护命中的观测不参与判定;
evaluateThreshold := decision.ReadingAccepted
if switched {
evaluateThreshold = switchedThresholdEvaluable(flow, decision.ReadingAccepted)
}
if observation.Metadata.Scene == constants.CardObservationScenePreActivationSettlement {
// 激活前结算只记已结算用量,不触发停复机评估;下一次正常轮询观测照常判定。
evaluateThreshold = false
}
if evaluateThreshold && s.channelThreshold != nil {
// 未切换卡只使用本次已接受的网关累计读数,异常下降保护命中的观测不参与判定;
// 已切换卡读数有效且未被重置日防旧值处理即判定,不再受卡表基线影响。
// 判定失败必须回滚整个流量事务,与既有流量事实保持同事务语义。
if err := s.channelThreshold.EvaluateInTx(ctx, tx, carrierthresholddomain.Evaluation{
CardID: card.ID, CarrierID: card.CarrierID,
ReadingMB: decision.LastGatewayReadingMB, ObservedAt: observation.Metadata.ObservedAt,
ReadingMB: thresholdReading(switched, flow, observation, decision), ObservedAt: observation.Metadata.ObservedAt,
// 观测来源是判定端获取触发来源的唯一位置:命中留痕据此区分自动判定、人工同步与回调。
Source: observation.Metadata.Source,
}); err != nil {
@@ -152,5 +180,28 @@ func (s *Service) ApplyTrafficObservation(ctx context.Context, observation domai
if s.cache != nil {
s.cache.Invalidate(ctx, observation.CardID)
}
if auditedCard != nil && !s.trafficCalc.Switched(auditedCard.ID) {
s.writeUnswitchedSnapshot(ctx, auditedCard, observation)
}
return decision, nil
}
// switchedThresholdEvaluable 判定已切换卡本次是否做通道阈值判定:读数有效且本次未被防旧值按 0 处理;
// 第一次切换观测仍按卡表基线算法是否接受读数判定。
func switchedThresholdEvaluable(flow *switchedFlowOutcome, oldAccepted bool) bool {
if flow == nil {
return false
}
if flow.FirstSwitch {
return oldAccepted
}
return !flow.StaleGuarded
}
// thresholdReading 返回通道阈值判定使用的网关累计读数:已切换卡(非第一次切换观测)用本次读数。
func thresholdReading(switched bool, flow *switchedFlowOutcome, observation domain.TrafficObservation, decision domain.TrafficDecision) float64 {
if switched && flow != nil && !flow.FirstSwitch {
return observation.GatewayReadingMB
}
return decision.LastGatewayReadingMB
}

View File

@@ -0,0 +1,171 @@
package cardobservation
import (
"time"
"context"
"strconv"
"strings"
"go.uber.org/zap"
"gorm.io/gorm"
"github.com/break/junhong_cmp_fiber/internal/model"
"github.com/break/junhong_cmp_fiber/pkg/config"
"github.com/break/junhong_cmp_fiber/pkg/constants"
)
// 灰度名单中扣减载体的前缀。
const (
trafficCalcCarrierDevice = "device"
trafficCalcCarrierIotCard = "iot_card"
)
// TrafficCalcPolicy 判定卡是否走日快照计量算法,以及未切换卡是否写快照。
//
// 名单在进程启动时展开为内存卡集合,观测事务内只查内存、不查库;API 与全部 Worker 必须使用一致的配置。
// 零值策略等同于 off 模式且不写快照。
type TrafficCalcPolicy struct {
mode string
cards map[uint]struct{}
snapshotEnabled bool
}
// Switched 判定卡是否走日快照算法。
func (p *TrafficCalcPolicy) Switched(cardID uint) bool {
if p == nil {
return false
}
switch p.mode {
case config.TrafficCalcModeFull:
return true
case config.TrafficCalcModeGray:
_, ok := p.cards[cardID]
return ok
default:
return false
}
}
// SnapshotEnabled 判定未切换卡是否写日快照。
func (p *TrafficCalcPolicy) SnapshotEnabled() bool {
return p != nil && p.snapshotEnabled
}
// BuildTrafficCalcPolicy 按配置展开灰度名单:device 展开为当前绑定的全部卡;iot_card 实际使用设备套餐时
// 记录错误并跳过该项(不阻止启动)。展开查询失败时整体退回 off 模式,保证判定出错一律走卡表基线算法。
func BuildTrafficCalcPolicy(ctx context.Context, db *gorm.DB, cfg config.TrafficCalcConfig, logger *zap.Logger) *TrafficCalcPolicy {
if logger == nil {
logger = zap.NewNop()
}
policy := &TrafficCalcPolicy{mode: cfg.Mode, cards: map[uint]struct{}{}, snapshotEnabled: cfg.FlowSnapshotEnabled}
if cfg.Mode != config.TrafficCalcModeGray {
logger.Info("卡流量计量模式", zap.String("mode", cfg.Mode), zap.Bool("flow_snapshot_enabled", cfg.FlowSnapshotEnabled))
return policy
}
for _, item := range strings.Split(cfg.Cards, ",") {
item = strings.TrimSpace(item)
if item == "" {
continue
}
if err := policy.expandItem(ctx, db, item, logger); err != nil {
logger.Error("展开卡流量计量灰度名单失败,本进程全部卡走卡表基线算法", zap.String("item", item), zap.Error(err))
return &TrafficCalcPolicy{mode: config.TrafficCalcModeOff, snapshotEnabled: cfg.FlowSnapshotEnabled}
}
}
logger.Info("卡流量计量灰度名单已展开", zap.String("cards", cfg.Cards), zap.Int("card_count", len(policy.cards)))
return policy
}
// expandItem 展开一项扣减载体;格式非法或单卡实际用设备套餐时只记录错误并跳过,查询失败时返回错误。
func (p *TrafficCalcPolicy) expandItem(ctx context.Context, db *gorm.DB, item string, logger *zap.Logger) error {
kind, rawID, found := strings.Cut(item, ":")
id, parseErr := strconv.ParseUint(strings.TrimSpace(rawID), 10, 64)
if !found || parseErr != nil || id == 0 {
logger.Error("卡流量计量灰度名单项格式非法,已跳过", zap.String("item", item))
return nil
}
switch strings.TrimSpace(kind) {
case trafficCalcCarrierDevice:
var cardIDs []uint
if err := db.WithContext(ctx).Model(&model.DeviceSimBinding{}).
Where("device_id = ? AND bind_status = ?", id, constants.BindStatusBound).
Pluck("iot_card_id", &cardIDs).Error; err != nil {
return err
}
for _, cardID := range cardIDs {
p.cards[cardID] = struct{}{}
}
return nil
case trafficCalcCarrierIotCard:
usesDevice, err := cardUsesDevicePackage(ctx, db, uint(id))
if err != nil {
return err
}
if usesDevice {
logger.Error("灰度名单中的单卡实际从设备套餐扣减,已跳过;请改为按设备整台切换", zap.String("item", item))
return nil
}
p.cards[uint(id)] = struct{}{}
return nil
default:
logger.Error("卡流量计量灰度名单项载体类型非法,已跳过", zap.String("item", item))
return nil
}
}
// cardUsesDevicePackage 与套餐扣减的载体解析同口径:卡自身没有生效中套餐、且当前绑定设备有生效中套餐时,
// 卡的流量扣到设备套餐上。
func cardUsesDevicePackage(ctx context.Context, db *gorm.DB, cardID uint) (bool, error) {
ownActive, err := countActivePackages(ctx, db, "iot_card_id = ?", cardID, time.Time{})
if err != nil || ownActive > 0 {
return false, err
}
deviceID, bound, err := activeBoundDeviceID(ctx, db, cardID)
if err != nil || !bound {
return false, err
}
deviceActive, err := countActivePackages(ctx, db, "device_id = ?", deviceID, time.Time{})
return deviceActive > 0, err
}
// hasActivePackageForCard 判定卡本次增量是否有可扣的生效中套餐:卡自身或当前绑定设备存在生效中套餐。
// observedAt 非零时只认激活时间不晚于观测时间的生效套餐(流量按观测时间归属):
// 观测早于套餐激活的用量属于激活前,只记已结算、不扣新套餐;激活时间为空的历史套餐视为早已生效。
func hasActivePackageForCard(ctx context.Context, db *gorm.DB, cardID uint, observedAt time.Time) (bool, error) {
ownActive, err := countActivePackages(ctx, db, "iot_card_id = ?", cardID, observedAt)
if err != nil || ownActive > 0 {
return ownActive > 0, err
}
deviceID, bound, err := activeBoundDeviceID(ctx, db, cardID)
if err != nil || !bound {
return false, err
}
deviceActive, err := countActivePackages(ctx, db, "device_id = ?", deviceID, observedAt)
return deviceActive > 0, err
}
func countActivePackages(ctx context.Context, db *gorm.DB, carrierCondition string, carrierID uint, observedAt time.Time) (int64, error) {
var count int64
query := db.WithContext(ctx).Model(&model.PackageUsage{}).
Where(carrierCondition, carrierID).
Where("status = ?", constants.PackageUsageStatusActive)
if !observedAt.IsZero() {
query = query.Where("activated_at IS NULL OR activated_at <= ?", observedAt)
}
err := query.Count(&count).Error
return count, err
}
func activeBoundDeviceID(ctx context.Context, db *gorm.DB, cardID uint) (uint, bool, error) {
var deviceIDs []uint
if err := db.WithContext(ctx).Model(&model.DeviceSimBinding{}).
Where("iot_card_id = ? AND bind_status = ?", cardID, constants.BindStatusBound).
Limit(1).Pluck("device_id", &deviceIDs).Error; err != nil {
return 0, false, err
}
if len(deviceIDs) == 0 {
return 0, false, nil
}
return deviceIDs[0], true, nil
}

View File

@@ -0,0 +1,150 @@
package cardobservation
import (
"context"
"time"
"go.uber.org/zap"
"gorm.io/gorm"
cardflow "github.com/break/junhong_cmp_fiber/internal/domain/cardflow"
domain "github.com/break/junhong_cmp_fiber/internal/domain/cardobservation"
"github.com/break/junhong_cmp_fiber/internal/model"
"github.com/break/junhong_cmp_fiber/internal/store/postgres"
"github.com/break/junhong_cmp_fiber/pkg/errors"
)
// unswitchedSnapshotTimeout 是未切换卡在观测事务提交后写快照的超时,超时只告警。
const unswitchedSnapshotTimeout = 3 * time.Second
// switchedFlowOutcome 是已切换卡一次观测的日快照计量结果。
type switchedFlowOutcome struct {
// EmitMB 是本次应写入 V1 流量事件的增量,0 表示不写事件。
EmitMB float64
// MonthMaxMB 是写入本次快照后的本月最大快照,用于覆盖卡表本月用量。
MonthMaxMB float64
// StaleGuarded 为 true 表示本次读数在重置日触发防旧值、按 0 计。
StaleGuarded bool
// FirstSwitch 为 true 表示这是卡的第一次切换观测(本次写入了切换水位)。
FirstSwitch bool
}
// snapshotResult 是一次快照换算与写入的结果。
type snapshotResult struct {
flowDate time.Time
conversion cardflow.Conversion
}
// writeSnapshot 按卡所属通道的重置日换算并原子写入当天快照;读数必须已通过 cardflow.ValidReading。
func (s *Service) writeSnapshot(ctx context.Context, db *gorm.DB, card *model.IotCard, readingMB float64, observedAt time.Time) (snapshotResult, error) {
resetDay := s.strictResetDay(ctx, db, card)
flowDate := cardflow.ShanghaiDate(observedAt)
days := postgres.NewCardFlowDayStore(db)
monthCtx, err := days.MonthContext(ctx, card.ID, flowDate, resetDay)
if err != nil {
return snapshotResult{}, errors.Wrap(errors.CodeDatabaseError, err, "查询卡日流量快照失败")
}
conversion := cardflow.Convert(readingMB, flowDate.Day(), resetDay, monthCtx)
if err := days.Upsert(ctx, postgres.CardFlowSnapshotWrite{
CardID: card.ID, FlowDate: flowDate, MonthUsedMB: conversion.MonthUsedMB,
VendorUsedMB: readingMB, StaleGuarded: conversion.StaleGuarded,
}); err != nil {
return snapshotResult{}, errors.Wrap(errors.CodeDatabaseError, err, "写入卡日流量快照失败")
}
return snapshotResult{flowDate: flowDate, conversion: conversion}, nil
}
// strictResetDay 在当前连接内按卡的 carrier_id 读取上游流量重置日,不使用调用方预先计算的值;
// 查询失败、为空或非法时按卡的运营商类型兜底(联通 27,其余 1)并记录日志。
func (s *Service) strictResetDay(ctx context.Context, db *gorm.DB, card *model.IotCard) int {
var days []int
queryErr := db.WithContext(ctx).Model(&model.Carrier{}).
Where("id = ?", card.CarrierID).Limit(1).Pluck("data_reset_day", &days).Error
queried := 0
if len(days) > 0 {
queried = days[0]
}
resetDay, fallback := cardflow.ResolveResetDay(queried, queryErr == nil && len(days) > 0, card.CarrierType)
if fallback {
s.warnLogger().Warn("通道上游流量重置日不可用,按运营商类型兜底",
zap.Uint("card_id", card.ID), zap.Uint("carrier_id", card.CarrierID),
zap.String("carrier_type", card.CarrierType), zap.Int("queried", queried),
zap.Int("reset_day", resetDay), zap.Error(queryErr))
}
return resetDay
}
// applySwitchedFlow 在观测事务内为已切换卡写快照、按月对账结算增量,并在第一次切换观测时写切换水位。
//
// oldDecision 是卡表基线算法对本次观测的结果:只在第一次切换观测时沿用其增量,此后不再使用。
func (s *Service) applySwitchedFlow(ctx context.Context, tx *gorm.DB, card *model.IotCard, observation domain.TrafficObservation, oldDecision domain.TrafficDecision) (*switchedFlowOutcome, error) {
readingMB := observation.GatewayReadingMB
snapshot, err := s.writeSnapshot(ctx, tx, card, readingMB, observation.Metadata.ObservedAt)
if err != nil {
return nil, err
}
days := postgres.NewCardFlowDayStore(tx)
monthMax, settled, err := days.MonthTotals(ctx, card.ID, snapshot.flowDate)
if err != nil {
return nil, errors.Wrap(errors.CodeDatabaseError, err, "查询卡本月流量结算失败")
}
hasPackage, pkgErr := hasActivePackageForCard(ctx, tx, card.ID, observation.Metadata.ObservedAt)
if pkgErr != nil {
// 判断出错按有套餐处理,保持不丢量。
s.warnLogger().Warn("判断卡是否有生效中套餐失败,按有套餐处理", zap.Uint("card_id", card.ID), zap.Error(pkgErr))
hasPackage = true
}
outcome := &switchedFlowOutcome{MonthMaxMB: monthMax, StaleGuarded: snapshot.conversion.StaleGuarded}
switches := postgres.NewCardFlowSwitchStore(tx)
watermark, err := switches.Get(ctx, card.ID)
if err != nil {
return nil, errors.Wrap(errors.CodeDatabaseError, err, "查询卡流量切换水位失败")
}
if watermark == nil {
return outcome, s.firstSwitch(ctx, tx, card, observation, oldDecision, snapshot.flowDate, monthMax, settled, hasPackage, outcome)
}
delta := cardflow.MonthlyDelta(monthMax, settled)
if delta <= 0 {
return outcome, nil
}
if err := days.AddSettled(ctx, card.ID, snapshot.flowDate, delta); err != nil {
return nil, errors.Wrap(errors.CodeDatabaseError, err, "结算卡流量增量失败")
}
if hasPackage {
outcome.EmitMB = delta
}
return outcome, nil
}
// firstSwitch 处理第一次切换观测:本次沿用卡表基线算法的增量,此前部分记为已结算,并写入切换水位。
func (s *Service) firstSwitch(ctx context.Context, tx *gorm.DB, card *model.IotCard, observation domain.TrafficObservation,
oldDecision domain.TrafficDecision, flowDate time.Time, monthMax, settled float64, hasPackage bool, outcome *switchedFlowOutcome) error {
outcome.FirstSwitch = true
if oldDecision.IncrementMB > 0 && hasPackage {
outcome.EmitMB = oldDecision.IncrementMB
}
if err := postgres.NewCardFlowDayStore(tx).AddSettled(ctx, card.ID, flowDate, cardflow.MonthlyDelta(monthMax, settled)); err != nil {
return errors.Wrap(errors.CodeDatabaseError, err, "结算卡流量切换前用量失败")
}
if err := postgres.NewCardFlowSwitchStore(tx).Create(ctx, &model.CardFlowSwitch{
IotCardID: card.ID, SwitchedAt: observation.Metadata.ObservedAt, MonthUsedMB: monthMax,
VendorUsedMB: observation.GatewayReadingMB, OldBaselineMB: card.LastGatewayReadingMB,
OldMonthUsageMB: card.CurrentMonthUsageMB, OldReadingAccepted: oldDecision.ReadingAccepted,
OldIncrementMB: oldDecision.IncrementMB,
}); err != nil {
return errors.Wrap(errors.CodeDatabaseError, err, "写入卡流量切换水位失败")
}
return nil
}
// writeUnswitchedSnapshot 在观测事务提交后为未切换卡写快照;失败只告警,不影响本次观测结果。
func (s *Service) writeUnswitchedSnapshot(ctx context.Context, card *model.IotCard, observation domain.TrafficObservation) {
if card == nil || !s.trafficCalc.SnapshotEnabled() || !cardflow.ValidReading(observation.GatewayReadingMB) {
return
}
writeCtx, cancel := context.WithTimeout(ctx, unswitchedSnapshotTimeout)
defer cancel()
if _, err := s.writeSnapshot(writeCtx, s.db, card, observation.GatewayReadingMB, observation.Metadata.ObservedAt); err != nil {
s.warnLogger().Warn("未切换卡写日流量快照失败,不影响本次观测", zap.Uint("card_id", card.ID), zap.Error(err))
}
}

View File

@@ -14,8 +14,8 @@ import (
// EvaluateInTx 在流量观测事务内判定通道阈值,达量时写周期锁并同事务写停机事件。
//
// 判定前提由调用方保证:只有「读数被接受」(ReadingAccepted)的观测才进入本方法,
// 异常下降保护命中的观测不参与判定。重复达量观测由部分唯一索引的 23505 识别为
// 判定前提由调用方保证:未切换日快照算法的卡只有「读数被接受」(ReadingAccepted)的观测才进入本方法,
// 异常下降保护命中的观测不参与判定;已切换的卡读数有效且未被重置日防旧值处理即进入本方法。重复达量观测由部分唯一索引的 23505 识别为
// 「该周期已处理」并幂等跳过:不重复建锁、不重复写停机事件,该冲突与流量基线 CAS 的
// CodeConflict 语义不同,绝不能混流。返回错误表示必须整体回滚(与既有流量事实同事务)。
func (s *Service) EvaluateInTx(ctx context.Context, tx *gorm.DB, evaluation domain.Evaluation) error {

View File

@@ -41,9 +41,13 @@ func (s *RefundOffsetService) ApplyInTx(ctx context.Context, tx *gorm.DB, source
if s == nil || tx == nil || s.audit == nil {
return errors.New(errors.CodeInternalError, "员工代收款退款冲销能力未配置")
}
if source.RefundID == 0 || source.OrderID == 0 || source.RefundAmount <= 0 {
if source.RefundID == 0 || source.OrderID == 0 || source.RefundAmount < 0 {
return errors.New(errors.CodeInvalidParam, "员工代收款退款冲销参数无效")
}
// 0 元退款只作废权益、不退钱:不冲减、不关闭账单、不写冲销记录,也不能让退款事务失败。
if source.RefundAmount == 0 {
return nil
}
var bill model.EmployeeCollectionBill
if err := tx.WithContext(ctx).Clauses(clause.Locking{Strength: "UPDATE"}).
Where("source_key = ?", employeecollectiondomain.OrderSourceKey(source.OrderID)).

View File

@@ -48,9 +48,9 @@ type pollingPriorityTriggerKey struct{}
// WithPollingPriorityTrigger 标记本次业务评估来自普通套餐轮询。
//
// 「资产无有效套餐」这一场景在停复机评估内部判定,而该评估入口被观测消费者、手动实名、保护期轮询、
// 「资产无有效套餐」这一场景在停复机评估内部判定,而该评估入口被观测消费者、手动实名、
// 既有兼容入口等多条调用链复用;只有普通套餐轮询来源才允许追加优先轮询请求,
// 否则会在观测序列、手动实名、保护期与退款等上下文凭空产生加急触发。
// 否则会在观测序列、手动实名与退款等上下文凭空产生加急触发。
func WithPollingPriorityTrigger(ctx context.Context) context.Context {
return context.WithValue(ctx, pollingPriorityTriggerKey{}, true)
}

View File

@@ -6,6 +6,7 @@ import (
"encoding/hex"
stdErrors "errors"
"fmt"
"strconv"
"strings"
"time"
@@ -332,6 +333,7 @@ func RequiredSceneFields(businessType string) []RequiredSceneField {
}
// RequiredRefundFields 返回退款提交固定与条件必需字段;快照方式编码缺失或未知时由表单构建拒绝。
// 客户收款信息退款要求客户收款信息与凭证必须映射并提交;0 元退款不退钱,两者不强制。
func RequiredRefundFields(snapshot map[string]any) []RequiredSceneField {
methodCode, ok := snapshot[constants.ApprovalFieldRefundMethodCode].(string)
methodCode = strings.TrimSpace(methodCode)
@@ -339,12 +341,22 @@ func RequiredRefundFields(snapshot map[string]any) []RequiredSceneField {
return []RequiredSceneField{{Code: constants.ApprovalFieldRefundMethodCode, Name: "退款方式编码"}}
}
result := []RequiredSceneField{{Code: constants.ApprovalFieldRefundMethod, Name: "退款方式", RequiredMapping: true}}
if methodCode == constants.RefundMethodCustomerAccount {
if methodCode == constants.RefundMethodCustomerAccount && !isZeroAmountRefund(snapshot) {
result = append(result, RequiredSceneField{Code: constants.ApprovalFieldCustomerAccountInfo, Name: "客户收款信息"}, RequiredSceneField{Code: constants.ApprovalFieldCustomerVoucherKey, Name: "客户收款凭证"})
}
return result
}
// isZeroAmountRefund 判断快照中的申请退款金额是否为 0;金额缺失或无法解析时按非 0 处理,保持强制校验。
func isZeroAmountRefund(snapshot map[string]any) bool {
raw, ok := snapshot[constants.ApprovalFieldRequestedRefundAmount].(string)
if !ok {
return false
}
amount, err := strconv.ParseFloat(strings.TrimSpace(raw), 64)
return err == nil && amount == 0
}
// approvalBusinessTypes 返回全部已登记审批业务场景,供只读字段汇总复用。
func approvalBusinessTypes() []string {
return []string{

View File

@@ -0,0 +1,121 @@
package bootstrap
import (
"context"
"strconv"
"strings"
cardObservationApp "github.com/break/junhong_cmp_fiber/internal/application/cardobservation"
"github.com/break/junhong_cmp_fiber/internal/polling"
"github.com/break/junhong_cmp_fiber/pkg/constants"
)
// carrierAwareSeriesDispatcher 是读入口观测分发的模式接缝:按运营商账户调度时把读入口改为按场景请求检查
// (同卡合并、冷却、受账户限流),不再起"立即、3 分钟、5 分钟"观测序列;旧分片调度时沿用原分发器。
type carrierAwareSeriesDispatcher struct {
legacy cardObservationApp.BestEffortSeriesDispatcher
requester *polling.CheckRequester
}
var _ cardObservationApp.BestEffortSeriesDispatcher = (*carrierAwareSeriesDispatcher)(nil)
// ReadEntryCheckScene 把读入口与设备控制的观测场景映射为请求检查场景;不认识的场景返回空串。
func ReadEntryCheckScene(observationScene string) string {
switch observationScene {
case constants.CardObservationSceneClientAssetRead:
return constants.PollingCheckSceneClientAssetDetail
case constants.CardObservationSceneAdminAssetRead:
return constants.PollingCheckSceneAdminAssetRealtime
case constants.CardObservationSceneClientRealnameLink, constants.CardObservationSceneAdminRealnameLink:
return constants.PollingCheckSceneRealnameLink
case constants.CardObservationSceneBusinessStop, constants.CardObservationSceneBusinessResume:
return constants.PollingCheckSceneStopResumeSucceeded
case constants.CardObservationScenePackageChanged:
return constants.PollingCheckScenePackageActivated
case constants.CardObservationSceneDeviceSwitchCard, constants.CardObservationSceneDeviceSwitchMode,
constants.CardObservationSceneDeviceReboot, constants.CardObservationSceneDeviceReset, constants.CardObservationSceneDeviceSetWiFi:
return constants.PollingCheckSceneDeviceOperation
default:
return ""
}
}
func (d *carrierAwareSeriesDispatcher) active(ctx context.Context) bool {
return d.requester != nil && d.requester.Active(ctx)
}
// Dispatch 读入口单卡(或设备)观测。
func (d *carrierAwareSeriesDispatcher) Dispatch(ctx context.Context, request cardObservationApp.SeriesRequest) {
if !d.active(ctx) {
d.legacy.Dispatch(ctx, request)
return
}
scene := ReadEntryCheckScene(request.Scene)
if scene == "" {
return
}
id, err := strconv.ParseUint(request.ResourceID, 10, 64)
if err != nil || id == 0 {
return
}
if request.ResourceType == constants.CardObservationResourceTypeDevice {
d.requester.RequestDevice(ctx, scene, uint(id))
return
}
d.requester.RequestCards(ctx, scene, []uint{uint(id)}, request.ExpectedValue)
}
// DispatchDeviceCards 设备有效绑定卡观测。
func (d *carrierAwareSeriesDispatcher) DispatchDeviceCards(ctx context.Context, request cardObservationApp.DeviceCardsSeriesRequest) {
if !d.active(ctx) {
d.legacy.DispatchDeviceCards(ctx, request)
return
}
if scene := ReadEntryCheckScene(request.Request.Scene); scene != "" {
d.requester.RequestDevice(ctx, scene, request.DeviceID)
}
}
// DispatchDeviceControl 设备控制成功后的观测:按设备操作场景为设备下各卡请求检查。
func (d *carrierAwareSeriesDispatcher) DispatchDeviceControl(ctx context.Context, request cardObservationApp.DeviceControlSeriesRequest) {
if !d.active(ctx) {
d.legacy.DispatchDeviceControl(ctx, request)
return
}
// 设备信息同步(query_device_info,非运营商上游)保持原观测,只把卡检查改为请求检查。
deviceInfo := request.Request
deviceInfo.ResourceType = constants.CardObservationResourceTypeDevice
deviceInfo.ResourceID = strconv.FormatUint(uint64(request.DeviceID), 10)
deviceInfo.SyncType = constants.CardObservationSyncTypeDeviceInfo
deviceInfo.ExpectedValue = strings.TrimSpace(request.TargetICCID)
d.legacy.Dispatch(ctx, deviceInfo)
cardIDs := append([]uint(nil), request.BoundCardIDs...)
for _, id := range []uint{request.SourceCardID, request.TargetCardID} {
if id != 0 && !containsUint(cardIDs, id) {
cardIDs = append(cardIDs, id)
}
}
if len(cardIDs) == 0 && request.DeviceID != 0 {
d.requester.RequestDevice(ctx, constants.PollingCheckSceneDeviceOperation, request.DeviceID)
return
}
d.requester.RequestCards(ctx, constants.PollingCheckSceneDeviceOperation, cardIDs, "")
}
// DispatchRealnameWithCapability 获取实名链接后的实名观测。
func (d *carrierAwareSeriesDispatcher) DispatchRealnameWithCapability(ctx context.Context, request cardObservationApp.RealnameCapabilitySeriesRequest) {
if !d.active(ctx) {
d.legacy.DispatchRealnameWithCapability(ctx, request)
return
}
d.Dispatch(ctx, request.Request)
}
func containsUint(values []uint, target uint) bool {
for _, value := range values {
if value == target {
return true
}
}
return false
}

View File

@@ -3,6 +3,7 @@ package bootstrap
import (
systemConfigApp "github.com/break/junhong_cmp_fiber/internal/application/systemconfig"
"github.com/break/junhong_cmp_fiber/internal/gateway"
"github.com/break/junhong_cmp_fiber/internal/infrastructure/gatewaylimit"
systemConfigInfra "github.com/break/junhong_cmp_fiber/internal/infrastructure/systemconfig"
"github.com/break/junhong_cmp_fiber/internal/service/verification"
"github.com/break/junhong_cmp_fiber/pkg/auth"
@@ -26,6 +27,7 @@ type Dependencies struct {
QueueClient *queue.Client // Asynq 任务队列客户端
StorageService *storage.Service // 对象存储服务(可选,配置缺失时为 nil)
GatewayClient *gateway.Client // Gateway API 客户端(可选,配置缺失时为 nil)
GatewayLimit *gatewaylimit.Runtime // 卡网关运营商账户限流组件(可选)
WechatPayment wechat.PaymentServiceInterface // 微信支付服务(可选)
SystemConfigRegistry *systemConfigInfra.Registry // 业务模块共享的受控配置注册表(可选)
SystemConfigAudit systemConfigApp.AuditWriter // 配置变更审计 Port;生产为空时装配统一 Writer

View File

@@ -21,6 +21,7 @@ import (
approvalInfra "github.com/break/junhong_cmp_fiber/internal/infrastructure/approval"
auditInfra "github.com/break/junhong_cmp_fiber/internal/infrastructure/audit"
"github.com/break/junhong_cmp_fiber/internal/infrastructure/carriercallback"
"github.com/break/junhong_cmp_fiber/internal/infrastructure/gatewaylimit"
"github.com/break/junhong_cmp_fiber/internal/infrastructure/integrationlog"
"github.com/break/junhong_cmp_fiber/internal/infrastructure/messaging/outbox"
notificationInfra "github.com/break/junhong_cmp_fiber/internal/infrastructure/notification"
@@ -28,6 +29,7 @@ import (
systemConfigInfra "github.com/break/junhong_cmp_fiber/internal/infrastructure/systemconfig"
wecomInfra "github.com/break/junhong_cmp_fiber/internal/infrastructure/wecom"
pollingPkg "github.com/break/junhong_cmp_fiber/internal/polling"
"github.com/break/junhong_cmp_fiber/internal/polling/carrierqueue"
agentRechargeQuery "github.com/break/junhong_cmp_fiber/internal/query/agentrecharge"
assetQuery "github.com/break/junhong_cmp_fiber/internal/query/asset"
auditQuery "github.com/break/junhong_cmp_fiber/internal/query/audit"
@@ -43,6 +45,7 @@ import (
packagetrafficalertquery "github.com/break/junhong_cmp_fiber/internal/query/packagetrafficalert"
priorityPollingQuery "github.com/break/junhong_cmp_fiber/internal/query/prioritypolling"
shopQuery "github.com/break/junhong_cmp_fiber/internal/query/shop"
"github.com/break/junhong_cmp_fiber/internal/query/stopresumerule"
systemConfigQuery "github.com/break/junhong_cmp_fiber/internal/query/systemconfig"
clientOrderSvc "github.com/break/junhong_cmp_fiber/internal/service/client_order"
"github.com/break/junhong_cmp_fiber/internal/service/paymentmethod"
@@ -120,6 +123,8 @@ func initHandlers(svc *services, deps *Dependencies) *Handlers {
}
registerCarrierCallbackConfigDefinitions(systemConfigRegistry, deps.Logger)
registerPaymentMethodConfigDefinitions(systemConfigRegistry, deps.Logger)
gatewaylimit.RegisterConfigDefinitions(systemConfigRegistry, deps.Logger)
carrierqueue.RegisterConfigDefinitions(systemConfigRegistry, deps.Logger)
systemConfigAlerts := systemConfigInfra.NewLogAlertSink(deps.Logger)
var systemConfigCache systemConfigInfra.Cache
if deps.Redis != nil {
@@ -301,6 +306,7 @@ func initHandlers(svc *services, deps *Dependencies) *Handlers {
PackageTrafficAlert: admin.NewPackageTrafficAlertHandler(svc.PackageTrafficAlertRule, packageTrafficAlertQuery, svc.ExportTask, validate),
AssetAutoRenewal: admin.NewAssetAutoRenewalConfigHandler(svc.AssetAutoRenewal, validate),
OperationsReport: admin.NewOperationsReportHandler(operationsReportQuery, svc.ExportTask, validate),
StopResumeRule: admin.NewStopResumeRuleHandler(stopresumerule.NewQuery(deps.DB), svc.ExportTask, validate),
ShopPackageBatchAllocation: admin.NewShopPackageBatchAllocationHandler(svc.ShopPackageBatchAllocation),
ShopPackageBatchPricing: admin.NewShopPackageBatchPricingHandler(svc.ShopPackageBatchPricing),
ShopSeriesGrant: admin.NewShopSeriesGrantHandler(svc.ShopSeriesGrant),
@@ -329,11 +335,14 @@ func initHandlers(svc *services, deps *Dependencies) *Handlers {
),
WeComApprovalCallback: wecomApprovalCallback,
PollingConfig: admin.NewPollingConfigHandler(svc.PollingConfig),
GatewayRateLimit: admin.NewGatewayRateLimitHandler(pollingSvcPkg.NewGatewayRateLimitService(deps.DB, auditInfra.NewWriter(auditInfra.NewRegistry(), nil), deps.GatewayLimit), deps.Redis, validate),
PollingConcurrency: admin.NewPollingConcurrencyHandler(svc.PollingConcurrency),
PollingMonitoring: admin.NewPollingMonitoringHandler(svc.PollingMonitoring),
PollingAlert: admin.NewPollingAlertHandler(svc.PollingAlert),
PollingCleanup: admin.NewPollingCleanupHandler(svc.PollingCleanup),
PollingManualTrigger: admin.NewPollingManualTriggerHandler(svc.PollingManualTrigger),
CarrierScheduler: admin.NewCarrierSchedulerHandler(pollingSvcPkg.NewCarrierSchedulerService(deps.DB, deps.Redis,
auditInfra.NewWriter(auditInfra.NewRegistry(), nil), carrierSchedulerConfig()), validate),
PollingMonitoring: admin.NewPollingMonitoringHandler(svc.PollingMonitoring),
PollingAlert: admin.NewPollingAlertHandler(svc.PollingAlert),
PollingCleanup: admin.NewPollingCleanupHandler(svc.PollingCleanup),
PollingManualTrigger: admin.NewPollingManualTriggerHandler(svc.PollingManualTrigger),
PollingPriority: admin.NewPriorityPollingHandler(
svc.PriorityPolling, priorityPollingQuery.NewQuery(deps.DB, pollingPriorityItemStore), validate,
),
@@ -401,3 +410,11 @@ func initHandlers(svc *services, deps *Dependencies) *Handlers {
AgentOpenAPI: openapiHandler.NewHandler(svc.AgentOpenAPI, validate),
}
}
// carrierSchedulerConfig 返回按运营商账户调度的 yaml 参数(供后台校验执行上限与展示默认值)。
func carrierSchedulerConfig() config.CarrierSchedulerConfig {
if cfg := config.Get(); cfg != nil {
return cfg.Polling.Carrier
}
return config.CarrierSchedulerConfig{}
}

View File

@@ -29,6 +29,7 @@ import (
walletinfra "github.com/break/junhong_cmp_fiber/internal/infrastructure/wallet"
wecomInfra "github.com/break/junhong_cmp_fiber/internal/infrastructure/wecom"
"github.com/break/junhong_cmp_fiber/internal/polling"
"github.com/break/junhong_cmp_fiber/internal/polling/carrierqueue"
assetAutoRenewalQuery "github.com/break/junhong_cmp_fiber/internal/query/assetautorenewal"
accountSvc "github.com/break/junhong_cmp_fiber/internal/service/account"
agentOpenAPISvc "github.com/break/junhong_cmp_fiber/internal/service/agent_open_api"
@@ -201,6 +202,12 @@ func initServices(s *stores, deps *Dependencies) *services {
cardObservationService.SetStateAuditWriter(iotCard)
// 运营商通道流量阈值达量判定嵌入流量观测事务:与卡流量事实同事务写周期锁与停机事件。
cardObservationService.SetChannelThresholdEvaluator(carrierThresholdService)
// 卡流量计量切换策略:启动时展开灰度名单为内存卡集合,API 与全部 Worker 必须使用一致配置。
var trafficCalcPolicy *cardObservationApp.TrafficCalcPolicy
if cfg := config.Get(); cfg != nil {
trafficCalcPolicy = cardObservationApp.BuildTrafficCalcPolicy(context.Background(), deps.DB, cfg.TrafficCalc, deps.Logger)
}
cardObservationService.SetTrafficCalcPolicy(trafficCalcPolicy, deps.Logger)
iotCard.SetCardObservationService(cardObservationService)
iotCard.SetSpeedTierIntegrationLog(integrationlog.NewRepository(deps.DB))
seriesCoordinator := cardObservationInfra.NewSeriesCoordinator(deps.Redis)
@@ -215,17 +222,26 @@ func initServices(s *stores, deps *Dependencies) *services {
cardObservationInfra.NewSeriesRunner(deps.DB, deps.GatewayClient, cardObservationService, seriesIntegration, auditWriter),
cardObservationInfra.NewSeriesAttemptLogger(seriesIntegration),
)
observationSeries := cardObservationInfra.NewBestEffortSeriesDispatcher(seriesTrigger, deps.Logger, s.DeviceSimBinding, s.Carrier)
// 读入口观测分发经模式接缝:按运营商账户调度时改为按场景请求检查,旧分片调度时沿用原分发器。
observationSeries := &carrierAwareSeriesDispatcher{legacy: cardObservationInfra.NewBestEffortSeriesDispatcher(seriesTrigger, deps.Logger, s.DeviceSimBinding, s.Carrier)}
iotCard.SetObservationSeriesDispatcher(observationSeries)
// 使用 PollingLifecycleService 替代 APICallback:通过分片队列准确操作(修复 api_callback.go 遗漏 protect 队列的 Bug3)
pollingConfigStore := postgres.NewPollingConfigStore(deps.DB)
pollingConfigMgr := polling.NewPollingConfigManager(pollingConfigStore, deps.Redis, deps.Logger)
pollingConfigMgr.SetTierResolver(polling.NewTierResolver(postgres.NewPackageUsageStore(deps.DB, deps.Redis),
postgres.NewDeviceSimBindingStore(deps.DB, deps.Redis), postgres.NewCarrierStore(deps.DB), deps.Logger))
if err := pollingConfigMgr.Load(context.Background()); err != nil {
deps.Logger.Warn("API 进程加载轮询配置失败", zap.Error(err))
}
pollingConfigMgr.Start(context.Background())
pollingQueueMgr := polling.NewPollingQueueManager(deps.Redis, constants.PollingShardCount, deps.Logger)
pollingLifecycleSvc := polling.NewPollingLifecycleService(pollingQueueMgr, pollingConfigMgr, s.IotCard, s.DeviceSimBinding, s.Device, deps.Logger)
// 按运营商账户调度:模式为 carrier 时 API 侧生命周期事件写入账户队列与卡到期信息。
carrierSettings := carrierqueue.NewDefaultSettingsProvider(deps.DB, deps.Redis, deps.Logger)
carrierStore := carrierqueue.NewStore(deps.Redis)
carrierScheduling := polling.NewCarrierScheduling(carrierStore, carrierSettings, pollingConfigMgr, deps.Logger)
pollingLifecycleSvc.SetCarrierScheduling(carrierScheduling)
observationSeries.requester = polling.NewCheckRequester(carrierScheduling, deps.DB, deps.Logger)
iotCard.SetPollingCallback(pollingLifecycleSvc)
// 创建支付配置服务(Order 和 Recharge 依赖)
wechatConfig := wechatConfigSvc.New(s.WechatConfig, s.Order, s.RechargeOrder, s.AgentRecharge, s.Payment, auditWriter, deps.Redis, deps.Logger)
@@ -244,6 +260,9 @@ func initServices(s *stores, deps *Dependencies) *services {
packageActivation.SetLifecycleAudit(auditWriter)
packageActivation.SetObservationSeriesEventWriter(observationSeriesEvents)
packageActivation.SetPriorityEventWriter(priorityEvents)
// 激活前流量结算:资产从无生效主套餐变为有生效主套餐前,对已切换日快照计量的卡查一次流量入账。
preActivationSettler := cardObservationInfra.NewPreActivationSettler(deps.DB, deps.GatewayClient, cardObservationService, postgres.NewCarrierStore(deps.DB), deps.Logger)
packageActivation.SetPreActivationSettler(preActivationSettler)
stopResumeService := iotCardSvc.NewStopResumeService(
deps.Redis,
@@ -257,7 +276,12 @@ func initServices(s *stores, deps *Dependencies) *services {
stopResumeService.SetObservationSeriesEventWriter(deps.DB, observationSeriesEvents)
stopResumeService.SetPriorityEventWriter(priorityEvents)
stopResumeService.SetUnifiedAudit(auditWriter, integrationlog.NewRepository(deps.DB))
// 持通道阈值锁的卡在周期内拒绝一切复机(自动、手动、保护期强制、机卡分离)。
if deps.GatewayLimit != nil {
stopResumeService.SetStopResumeRecorder(deps.GatewayLimit.Recorder)
deps.GatewayLimit.TerminalPolling.SetAuditWriter(auditWriter)
stopResumeService.SetTerminalPollingDisabler(deps.GatewayLimit.TerminalPolling)
}
// 持通道阈值锁的卡在周期内拒绝一切复机(自动、手动、机卡分离)。
stopResumeService.SetChannelThresholdLockGuard(carrierThresholdService)
iotCard.SetRealnameActivator(packageActivation)
iotCard.SetStopResumeService(stopResumeService)
@@ -287,6 +311,10 @@ func initServices(s *stores, deps *Dependencies) *services {
device.SetGatewayIntegrationLog(integrationlog.NewRepository(deps.DB))
device.SetObservationSeriesEventWriter(observationSeriesEvents)
device.SetObservationSeriesDispatcher(observationSeries)
if deps.GatewayLimit != nil {
device.SetStopResumeRecorder(deps.GatewayLimit.Recorder)
device.SetTerminalPollingDisabler(deps.GatewayLimit.TerminalPolling)
}
operationPassword := operationPasswordSvc.New(deps.Redis)
shopCommission := shopCommissionSvc.New(s.Shop, s.Account, s.AgentWallet, s.CommissionWithdrawalRequest, s.CommissionWithdrawalSetting, s.CommissionRecord, s.AgentWalletTransaction, deps.DB, deps.Logger)
shopCommission.SetAuditWriter(auditWriter)
@@ -298,6 +326,7 @@ func initServices(s *stores, deps *Dependencies) *services {
// 员工代收款建账用例在订单、充值入账与退款冲销的事务内复用同一实例。
employeeCollectionBillCreation := employeecollectionApp.NewBillCreationService(auditWriter)
orderService.SetEmployeeCollectionBillCreation(employeeCollectionBillCreation)
orderService.SetPreActivationSettler(preActivationSettler)
orderService.SetResumeCallback(stopResumeService)
orderService.SetLifecycleAudit(auditWriter)
orderService.SetPaymentIntegrationLog(integrationlog.NewRepository(deps.DB))
@@ -456,6 +485,7 @@ func initServices(s *stores, deps *Dependencies) *services {
commissionCalculation.SetAuditWriter(auditWriter)
pollingConfigService := pollingSvc.NewConfigService(s.PollingConfig, deps.Redis, deps.Logger)
pollingConfigService.SetAudit(deps.DB, auditWriter)
pollingConfigService.SetTierDeps(pollingConfigMgr, deps.QueueClient)
pollingConcurrencyService := pollingSvc.NewConcurrencyService(s.PollingConcurrencyConfig, deps.Redis)
pollingConcurrencyService.SetAudit(deps.DB, auditWriter)
pollingAlertService := pollingSvc.NewAlertService(s.PollingAlertRule, s.PollingAlertHistory, deps.Redis, deps.Logger)
@@ -550,7 +580,7 @@ func initServices(s *stores, deps *Dependencies) *services {
AgentRechargePaymentConfirm: agentRechargePaymentConfirm,
AgentRechargeVoucherOCR: agentRechargeVoucherOCR,
PackageActivation: packageActivation,
TrafficQuery: trafficSvc.NewQueryService(deps.Redis, s.CardDailyUsage),
TrafficQuery: trafficSvc.NewQueryService(deps.Redis, s.CardDailyUsage).WithFlowSnapshots(trafficCalcPolicy.Switched, postgres.NewCardFlowDayStore(deps.DB)),
OperationPassword: operationPassword,
AgentOpenAPI: agentOpenAPI,
Refund: refundService,

View File

@@ -64,6 +64,8 @@ type Handlers struct {
CUCCRealnameRemovalCallback *callback.CUCCRealnameRemovalHandler
WeComApprovalCallback *callback.WeComApprovalHandler
PollingConfig *admin.PollingConfigHandler
GatewayRateLimit *admin.GatewayRateLimitHandler
CarrierScheduler *admin.CarrierSchedulerHandler
PollingConcurrency *admin.PollingConcurrencyHandler
PollingMonitoring *admin.PollingMonitoringHandler
PollingAlert *admin.PollingAlertHandler
@@ -86,6 +88,7 @@ type Handlers struct {
PackageTrafficAlert *admin.PackageTrafficAlertHandler
AssetAutoRenewal *admin.AssetAutoRenewalConfigHandler
OperationsReport *admin.OperationsReportHandler
StopResumeRule *admin.StopResumeRuleHandler
ClientWechat *app.ClientWechatHandler
SuperAdmin *admin.SuperAdminHandler
SystemConfig *admin.SystemConfigHandler

View File

@@ -7,6 +7,7 @@ import (
"gorm.io/gorm"
"github.com/break/junhong_cmp_fiber/internal/gateway"
"github.com/break/junhong_cmp_fiber/internal/infrastructure/gatewaylimit"
"github.com/break/junhong_cmp_fiber/pkg/queue"
"github.com/break/junhong_cmp_fiber/pkg/storage"
)
@@ -16,10 +17,11 @@ type WorkerDependencies struct {
DB *gorm.DB
Redis *redis.Client
Logger *zap.Logger
AsynqClient *asynq.Client // Worker 特有:用于 Scheduler 提交任务
QueueClient *queue.Client // 统一业务任务客户端,用于 Worker 内产生后续任务
StorageService *storage.Service // 对象存储(可选)
GatewayClient *gateway.Client // Gateway 客户端(可选)
AsynqClient *asynq.Client // Worker 特有:用于 Scheduler 提交任务
QueueClient *queue.Client // 统一业务任务客户端,用于 Worker 内产生后续任务
StorageService *storage.Service // 对象存储(可选)
GatewayClient *gateway.Client // Gateway 客户端(可选)
GatewayLimit *gatewaylimit.Runtime // 卡网关运营商账户限流组件(可选)
}
// WorkerBootstrapResult Worker Bootstrap 初始化结果

View File

@@ -1,6 +1,11 @@
package bootstrap
import (
"context"
"github.com/break/junhong_cmp_fiber/internal/store/postgres"
"github.com/break/junhong_cmp_fiber/pkg/config"
assetAutoRenewalApp "github.com/break/junhong_cmp_fiber/internal/application/assetautorenewal"
cardObservationApp "github.com/break/junhong_cmp_fiber/internal/application/cardobservation"
carrierThresholdApp "github.com/break/junhong_cmp_fiber/internal/application/carrierthreshold"
@@ -120,6 +125,14 @@ func initWorkerServices(stores *queue.WorkerStores, deps *WorkerDependencies) *q
cardObservationService.SetStateAuditWriter(iotCardAuditService)
// 运营商通道流量阈值达量判定嵌入流量观测事务:与卡流量事实同事务写周期锁与停机事件。
cardObservationService.SetChannelThresholdEvaluator(carrierThresholdService)
// 卡流量计量切换策略:启动时展开灰度名单为内存卡集合,API 与全部 Worker 必须使用一致配置。
if cfg := config.Get(); cfg != nil {
cardObservationService.SetTrafficCalcPolicy(
cardObservationApp.BuildTrafficCalcPolicy(context.Background(), deps.DB, cfg.TrafficCalc, deps.Logger), deps.Logger)
}
// 激活前流量结算:与 API 进程一致,注入激活服务与订单服务。
preActivationSettler := cardObservationInfra.NewPreActivationSettler(deps.DB, deps.GatewayClient, cardObservationService, postgres.NewCarrierStore(deps.DB), deps.Logger)
activationService.SetPreActivationSettler(preActivationSettler)
cardObservationIntegration := integrationlog.NewRepository(deps.DB)
cardObservationSeriesCoordinator := cardObservationInfra.NewSeriesCoordinator(deps.Redis)
cardObservationSeriesService := cardObservationApp.NewSeriesAttemptService(
@@ -157,6 +170,7 @@ func initWorkerServices(stores *queue.WorkerStores, deps *WorkerDependencies) *q
)
// 批量订购线下支付建单需在订单事务内生成员工代收款账单,与 API 进程保持一致。
orderService.SetEmployeeCollectionBillCreation(employeecollectionApp.NewBillCreationService(auditWriter))
orderService.SetPreActivationSettler(preActivationSettler)
orderService.SetLifecycleAudit(auditWriter)
orderService.SetPaymentIntegrationLog(integrationlog.NewRepository(deps.DB))
walletOutbox := outbox.NewRepository()
@@ -178,7 +192,12 @@ func initWorkerServices(stores *queue.WorkerStores, deps *WorkerDependencies) *q
stopResumeService.SetObservationSeriesEventWriter(deps.DB, observationSeriesEvents)
stopResumeService.SetPriorityEventWriter(priorityEvents)
stopResumeService.SetUnifiedAudit(auditWriter, integrationlog.NewRepository(deps.DB))
// 持通道阈值锁的卡在周期内拒绝一切复机(自动、手动、保护期强制、机卡分离)。
if deps.GatewayLimit != nil {
stopResumeService.SetStopResumeRecorder(deps.GatewayLimit.Recorder)
deps.GatewayLimit.TerminalPolling.SetAuditWriter(auditWriter)
stopResumeService.SetTerminalPollingDisabler(deps.GatewayLimit.TerminalPolling)
}
// 持通道阈值锁的卡在周期内拒绝一切复机(自动、手动、机卡分离)。
stopResumeService.SetChannelThresholdLockGuard(carrierThresholdService)
// 停复机执行端口复用既有停复机服务作为唯一事实源:消费者与两个计划任务共用同一用例实例。
carrierThresholdService.SetCommander(carrierThresholdInfra.NewCardCommander(stopResumeService))

View File

@@ -0,0 +1,5 @@
// Package cardflow 提供卡日流量快照计量的纯领域规则:自然月换算、重置日防旧值、按月对账增量与每天用量。
//
// 规则照搬老卡管:每卡每天一条「本月自然月累计」快照,任意区间用量 = 区间内最大快照 − 区间开始前最大快照。
// 本包不依赖数据库、Redis 或外部 SDK,取值由调用方从快照存储查询后传入。
package cardflow

View File

@@ -0,0 +1,126 @@
package cardflow
import (
"math"
"strings"
"time"
)
// 计量规则常量。
const (
// StaleGuardRatio 是重置日防旧值阈值:重置日读数超过比较基准的该倍数时按 0 计。
StaleGuardRatio = 0.9
// MinResetDay 与 MaxResetDay 是通道上游流量重置日的合法范围。
MinResetDay = 1
MaxResetDay = 28
// FallbackResetDayCUCC 是联通卡重置日查询失败时的兜底值。
FallbackResetDayCUCC = 27
// FallbackResetDayDefault 是非联通卡重置日查询失败时的兜底值。
FallbackResetDayDefault = 1
// carrierTypeCUCC 是联通运营商类型编码。
carrierTypeCUCC = "CUCC"
)
// shanghaiLocation 是快照日期与周期边界的固定时区。
var shanghaiLocation = time.FixedZone("Asia/Shanghai", 8*60*60)
// ShanghaiDate 返回观测时刻对应的上海日期(当天 0 点)。
func ShanghaiDate(t time.Time) time.Time {
local := t.In(shanghaiLocation)
return time.Date(local.Year(), local.Month(), local.Day(), 0, 0, 0, 0, shanghaiLocation)
}
// MonthStart 返回日期所在自然月 1 号(上海时区)。
func MonthStart(date time.Time) time.Time {
local := date.In(shanghaiLocation)
return time.Date(local.Year(), local.Month(), 1, 0, 0, 0, 0, shanghaiLocation)
}
// ResolveResetDay 返回可用的重置日;查询失败或取值非法时按运营商类型兜底,fallback 为 true 表示使用了兜底值。
func ResolveResetDay(queried int, queryOK bool, carrierType string) (resetDay int, fallback bool) {
if queryOK && queried >= MinResetDay && queried <= MaxResetDay {
return queried, false
}
if strings.EqualFold(strings.TrimSpace(carrierType), carrierTypeCUCC) {
return FallbackResetDayCUCC, true
}
return FallbackResetDayDefault, true
}
// MonthContext 是换算与防旧值需要的已有快照取值,均为自然月累计(MB)。
type MonthContext struct {
// ThisMonthBeforeResetMax 是本月 1 号~重置日前一天的最大快照。
ThisMonthBeforeResetMax float64
// PrevMonthMax 是上月最大快照。
PrevMonthMax float64
// PrevMonthBeforeResetMax 是上月 1 号~重置日前一天的最大快照。
PrevMonthBeforeResetMax float64
}
// Conversion 是一次读数换算为自然月累计的结果。
type Conversion struct {
// MonthUsedMB 是换算后的本月自然月累计。
MonthUsedMB float64
// StaleGuarded 为 true 表示本次读数在重置日触发了防旧值、按 0 参与换算。
StaleGuarded bool
}
// Convert 把运营商周期读数换算为自然月累计,并在重置日当天应用防旧值。
//
// 重置日为 1 时运营商周期即自然月;重置日大于 1 时:日号 ≥ 重置日累加本月重置日前的最大快照,
// 日号 < 重置日扣除上月重置日及以后的用量(上月最大 − 上月重置日前最大),结果小于 0 按 0。
func Convert(readingMB float64, day int, resetDay int, ctx MonthContext) Conversion {
reading := readingMB
guarded := isStale(readingMB, day, resetDay, ctx)
if guarded {
reading = 0
}
var month float64
switch {
case resetDay <= 1:
month = reading
case day >= resetDay:
month = reading + ctx.ThisMonthBeforeResetMax
default:
month = reading - (ctx.PrevMonthMax - ctx.PrevMonthBeforeResetMax)
}
return Conversion{MonthUsedMB: nonNegative(round3(month)), StaleGuarded: guarded}
}
// isStale 判断重置日当天的读数是否仍接近上周期总量。
func isStale(readingMB float64, day int, resetDay int, ctx MonthContext) bool {
if day != resetDay || readingMB <= 0 {
return false
}
if resetDay <= 1 {
return readingMB > ctx.PrevMonthMax*StaleGuardRatio
}
return ctx.ThisMonthBeforeResetMax > 0 && readingMB > ctx.ThisMonthBeforeResetMax*StaleGuardRatio
}
// MonthlyDelta 返回按月对账的待结算增量:本月最大快照 − 本月已结算合计,不大于 0 时返回 0。
func MonthlyDelta(monthMaxMB float64, settledSumMB float64) float64 {
return nonNegative(round3(monthMaxMB - settledSumMB))
}
// DailyUsage 返回某天用量:当天最大快照 − 本月此前最大快照,小于 0 按 0;本月第一条快照时 priorMax 传 0。
func DailyUsage(dayMaxMB float64, priorMaxMB float64) float64 {
return nonNegative(round3(dayMaxMB - priorMaxMB))
}
// ValidReading 判断运营商读数能否写入快照:必须为有限正数。
func ValidReading(readingMB float64) bool {
return !math.IsNaN(readingMB) && !math.IsInf(readingMB, 0) && readingMB > 0
}
func nonNegative(v float64) float64 {
if v < 0 {
return 0
}
return v
}
// round3 按快照列精度(三位小数)取整,避免浮点误差产生极小的虚假增量。
func round3(v float64) float64 {
return math.Round(v*1000) / 1000
}

View File

@@ -21,7 +21,6 @@ type CardNetworkSnapshot struct {
CardID uint
NetworkStatus int
StopReason string
IsStandalone bool
EnablePolling bool
}
@@ -39,7 +38,7 @@ type NetworkDecision struct {
StopPolling bool
}
// ApplyNetwork 应用 Gateway 状态映射、运营商停机原因和独立风险卡规则。
// ApplyNetwork 应用 Gateway 状态映射、运营商停机原因和风险终止卡关闭轮询规则。
func ApplyNetwork(snapshot CardNetworkSnapshot, observation NetworkObservation) (NetworkDecision, error) {
if err := validateObservationMetadata(observation.CardID, observation.Metadata); err != nil {
return NetworkDecision{}, err
@@ -53,7 +52,7 @@ func ApplyNetwork(snapshot CardNetworkSnapshot, observation NetworkObservation)
decision := NetworkDecision{
StatusKnown: known, BeforeStatus: snapshot.NetworkStatus, AfterStatus: snapshot.NetworkStatus,
GatewayExtend: extend, GatewayIMEI: imei, UpdateIMEI: imei != "", StopReason: snapshot.StopReason,
StopPolling: snapshot.EnablePolling && ShouldStopPollingForRisk(snapshot.IsStandalone, extend),
StopPolling: snapshot.EnablePolling && ShouldStopPollingForRisk(extend),
}
if !known {
return decision, nil
@@ -84,7 +83,8 @@ func MapGatewayNetworkStatus(cardStatus, extend string) (int, bool) {
return constants.NetworkStatusOffline, false
}
// ShouldStopPollingForRisk 判断独立卡是否命中运营商风险终止状态。
func ShouldStopPollingForRisk(isStandalone bool, extend string) bool {
return isStandalone && (extend == constants.GatewayCardExtendRiskStop || extend == constants.GatewayCardExtendCancelled)
// ShouldStopPollingForRisk 判断卡是否命中运营商风险终止状态(风险停机、已销户)。
// 不论是否绑定设备都关闭该卡自身轮询,不级联设备与设备下其他卡。
func ShouldStopPollingForRisk(extend string) bool {
return extend == constants.GatewayCardExtendRiskStop || extend == constants.GatewayCardExtendCancelled
}

View File

@@ -0,0 +1,137 @@
// Package stopresume 定义停复机新规则的纯判断:输入卡与资产的当前事实,输出目标动作,不读写任何外部状态。
//
// 新规则(finalize-unified-stop-resume-rule):
// - 卡或其有效绑定设备暂停轮询时不自动停复机;
// - 开机卡命中任一停机条件即停机:无套餐、流量用完、通道开启实名管控且卡未实名;
// - 停机卡同时满足有套餐、流量未用完、实名条件时复机,不看停机原因;持本周期通道阈值锁的卡拒绝复机;
// - 卡所属通道不存在(无法判断实名管控)时本次不动作。
package stopresume
import "github.com/break/junhong_cmp_fiber/pkg/constants"
// 判断结果。
const (
ResultWillResume = "will_resume" // 将复机
ResultWillStop = "will_stop" // 将停机
ResultNoAction = "no_action" // 不动作
)
// 不动作原因。
const (
NoActionPollingSuspended = "polling_suspended" // 卡或所在设备暂停轮询
NoActionConditionsMet = "conditions_met" // 开机卡满足开机条件
NoActionNoPackage = "no_package" // 停机卡无套餐(与停机原因同值,共用名称)
NoActionTrafficExhausted = "traffic_exhausted" // 停机卡流量已用完
NoActionNotRealname = "not_realname" // 停机卡所属通道开启实名管控且卡未实名
NoActionThresholdLocked = "threshold_locked" // 停机卡持本周期通道阈值锁
NoActionCarrierMissing = "carrier_missing" // 卡所属通道不存在,无法判断实名管控
NoActionUnknownNetwork = "unknown_network" // 卡网络状态既非开机也非停机
)
// Facts 是判断一张卡所需的全部事实;套餐与流量按绑定设备(有有效绑定时)或卡自身计算。
type Facts struct {
NetworkStatus int
PollingSuspended bool // 卡自身或其有效绑定设备暂停轮询
HasValidPackage bool // 存在生效中的套餐
TrafficExhausted bool // 生效或已用完的套餐全部达到流量上限
RealnameVerified bool
CarrierKnown bool // 卡所属通道存在
RealnameControl bool // 卡所属通道开启实名管控(可为假设值)
ThresholdLocked bool // 持本周期通道阈值锁
CurrentStopReason string
}
// Decision 是一张卡的判断结果。
type Decision struct {
Result string // ResultWillResume / ResultWillStop / ResultNoAction
Reason string // 将停机时为目标停机原因;不动作时为不动作原因;将复机时为空
StopReason string // 将停机时的目标停机原因
}
// Decide 按新规则判断卡的目标动作。
func Decide(f Facts) Decision {
if f.PollingSuspended {
return noAction(NoActionPollingSuspended)
}
if !f.CarrierKnown {
return noAction(NoActionCarrierMissing)
}
realnameOK := !f.RealnameControl || f.RealnameVerified
switch f.NetworkStatus {
case constants.NetworkStatusOnline:
// 停机原因优先级:流量用完 > 无套餐(仅在未用完时判断)> 未实名,与评估主流程一致。
switch {
case f.TrafficExhausted:
return stop(constants.StopReasonTrafficExhausted)
case !f.HasValidPackage:
return stop(constants.StopReasonNoPackage)
case !realnameOK:
return stop(constants.StopReasonNotRealname)
default:
return noAction(NoActionConditionsMet)
}
case constants.NetworkStatusOffline:
switch {
case !f.HasValidPackage:
return noAction(NoActionNoPackage)
case f.TrafficExhausted:
return noAction(NoActionTrafficExhausted)
case !realnameOK:
return noAction(NoActionNotRealname)
case f.ThresholdLocked:
return noAction(NoActionThresholdLocked)
default:
return Decision{Result: ResultWillResume}
}
default:
return noAction(NoActionUnknownNetwork)
}
}
func stop(reason string) Decision {
return Decision{Result: ResultWillStop, Reason: reason, StopReason: reason}
}
func noAction(reason string) Decision {
return Decision{Result: ResultNoAction, Reason: reason}
}
// ResultName 返回判断结果中文名称。
func ResultName(result string) string {
switch result {
case ResultWillResume:
return "将复机"
case ResultWillStop:
return "将停机"
case ResultNoAction:
return "不动作"
default:
return result
}
}
// ReasonName 返回停机原因或不动作原因的中文名称。
func ReasonName(reason string) string {
switch reason {
case "":
return ""
case NoActionPollingSuspended:
return "暂停轮询"
case NoActionConditionsMet:
return "满足开机条件"
case NoActionNoPackage:
return "无套餐"
case NoActionTrafficExhausted:
return "流量已用完"
case NoActionNotRealname:
return "未实名(通道开启实名管控)"
case NoActionThresholdLocked:
return "持通道阈值锁"
case NoActionCarrierMissing:
return "所属通道不存在"
case NoActionUnknownNetwork:
return "网络状态未知"
default:
return reason
}
}

View File

@@ -41,6 +41,7 @@ func NewDefaultRegistry(db *gorm.DB) *Registry {
NewExpiringAssetDataSource(db),
NewOperationsActivationDataSource(db),
NewOperationsRenewalDataSource(db),
NewStopResumeRuleDataSource(db),
)
}
@@ -81,7 +82,8 @@ func IsSupportedScene(scene string) bool {
constants.ExportTaskScenePackageTrafficAlert,
constants.ExportTaskSceneExpiringAsset,
constants.ExportTaskSceneOperationsActivation,
constants.ExportTaskSceneOperationsRenewal:
constants.ExportTaskSceneOperationsRenewal,
constants.ExportTaskSceneStopResumeRuleDryRun:
return true
default:
return false

View File

@@ -0,0 +1,98 @@
package exporter
import (
"context"
"strconv"
"gorm.io/gorm"
"github.com/break/junhong_cmp_fiber/internal/query/stopresumerule"
"github.com/break/junhong_cmp_fiber/pkg/constants"
)
// stopResumeRuleHeaders 停复机规则演练导出表头。
var stopResumeRuleHeaders = []string{
"卡ID", "ICCID", "运营商账户ID", "运营商账户", "绑定设备ID", "当前网络状态", "当前停机原因", "卡类别",
"满足实名条件", "判断结果", "原因",
}
// StopResumeRuleDataSource 停复机规则演练明细导出数据源:按卡 ID 升序分页取卡并逐张按新规则判断,只读。
// 仅超级管理员与平台账号可导出:受控入口已做角色门禁,这里再按任务内冻结的账号类型复核一次。
type StopResumeRuleDataSource struct {
query *stopresumerule.Query
}
// NewStopResumeRuleDataSource 创建停复机规则演练导出数据源。
func NewStopResumeRuleDataSource(db *gorm.DB) *StopResumeRuleDataSource {
return &StopResumeRuleDataSource{query: stopresumerule.NewQuery(db)}
}
// Scene 返回导出场景编码。
func (s *StopResumeRuleDataSource) Scene() string {
return constants.ExportTaskSceneStopResumeRuleDryRun
}
// Count 统计参与演练的卡数。
func (s *StopResumeRuleDataSource) Count(ctx context.Context, params ExportParams) (int, error) {
filter, err := s.filter(params)
if err != nil {
return 0, err
}
total, err := s.query.Count(ctx, filter)
return int(total), err
}
// Headers 返回导出表头。
func (s *StopResumeRuleDataSource) Headers(ctx context.Context, params ExportParams) ([]string, error) {
if err := stopresumerule.EnsureAllowed(params.UserType); err != nil {
return nil, err
}
return stopResumeRuleHeaders, nil
}
// Fetch 按 offset/limit 返回演练明细行。
func (s *StopResumeRuleDataSource) Fetch(ctx context.Context, params ExportParams, offset, limit int) ([][]string, error) {
filter, err := s.filter(params)
if err != nil {
return nil, err
}
rows, err := s.query.Page(ctx, filter, offset, limit)
if err != nil {
return nil, err
}
result := make([][]string, 0, len(rows))
for index := range rows {
item := stopresumerule.ToItem(&rows[index])
network := "停机"
if item.NetworkStatus == constants.NetworkStatusOnline {
network = "开机"
}
device := ""
if item.DeviceID != 0 {
device = strconv.FormatUint(uint64(item.DeviceID), 10)
}
realname := "否"
if item.RealnameOK {
realname = "是"
}
result = append(result, []string{
strconv.FormatUint(uint64(item.CardID), 10), item.ICCID, strconv.FormatUint(uint64(item.CarrierID), 10),
item.CarrierName, device, network, item.StopReasonName, item.CardCategory, realname, item.ResultName, item.ReasonName,
})
}
return result, nil
}
func (s *StopResumeRuleDataSource) filter(params ExportParams) (stopresumerule.Filter, error) {
if err := stopresumerule.EnsureAllowed(params.UserType); err != nil {
return stopresumerule.Filter{}, err
}
var carrierID *uint
if id, ok := filterUint(params.Filters, "carrier_id"); ok && id > 0 {
carrierID = &id
}
stopReason, _ := filterString(params.Filters, "stop_reason")
on, _ := filterString(params.Filters, "assume_control_on")
off, _ := filterString(params.Filters, "assume_control_off")
return stopresumerule.BuildFilter(carrierID, stopReason, on, off)
}

View File

@@ -0,0 +1,158 @@
package gateway
import (
"context"
stderrors "errors"
"time"
"unicode/utf8"
"github.com/break/junhong_cmp_fiber/pkg/constants"
"github.com/break/junhong_cmp_fiber/pkg/errors"
)
// GatewayError 是网关调用失败的结构化错误。
// 它包裹原有的 *errors.AppError:Error() 文本与错误码保持不变,调用方仍可 errors.As 取 AppError;
// 需要类别、上游业务码、上游信息与耗时的调用方用 errors.As 取 *GatewayError。
type GatewayError struct {
Category string // 错误类别,见 constants.GatewayError*
ProviderCode int // 网关业务码(仅上游业务错误时非 0)
ProviderMessage string // 网关返回信息(已截断,仅上游业务错误时非空)
HTTPStatus int // HTTP 状态码(仅 HTTP 异常时非 0)
Duration time.Duration // 本次 HTTP 尝试耗时
app *errors.AppError
}
// Error 返回与原 AppError 一致的文本。
func (e *GatewayError) Error() string {
if e == nil || e.app == nil {
return "Gateway 调用失败"
}
return e.app.Error()
}
// Unwrap 返回原 AppError,保持错误码判断兼容。
func (e *GatewayError) Unwrap() error {
if e == nil || e.app == nil {
return nil
}
return e.app
}
func newGatewayError(category string, app *errors.AppError) *GatewayError {
return &GatewayError{Category: category, app: app}
}
// AsGatewayError 从错误链中取出结构化网关错误。
func AsGatewayError(err error) (*GatewayError, bool) {
var gwErr *GatewayError
if stderrors.As(err, &gwErr) && gwErr != nil {
return gwErr, true
}
return nil, false
}
// IsRateLimited 判断错误是否为本系统运营商账户限流拒绝(未发出请求)。
func IsRateLimited(err error) bool {
gwErr, ok := AsGatewayError(err)
return ok && gwErr.Category == constants.GatewayErrorRateLimited
}
// NewRateLimitedError 创建限流拒绝错误;由限流守卫在拿不到名额时返回。
func NewRateLimitedError() error {
return newGatewayError(constants.GatewayErrorRateLimited,
errors.New(errors.CodeTooManyRequests, "运营商账户网关调用已达限流上限,请稍后重试"))
}
// truncateProviderMessage 按字符截断上游信息,避免超长内容落库或展示。
func truncateProviderMessage(msg string) string {
if utf8.RuneCountInString(msg) <= constants.GatewayProviderMessageMaxLen {
return msg
}
runes := []rune(msg)
return string(runes[:constants.GatewayProviderMessageMaxLen])
}
// CallInfo 描述一次卡网关 HTTP 尝试,供调用守卫限流与统计。
type CallInfo struct {
Operation string // 卡网关操作,见 constants.GatewayOp*
CardNo string // 卡号(ICCID)
CarrierID uint // 调用方已知的运营商账户 ID(本系统 tb_carrier.id),0 表示由守卫按卡号解析
ExemptSource string // 豁免限流来源,空表示受限流约束
Attempt int // 本次方法调用内的尝试序号,从 1 开始
}
// CallGuard 是卡网关调用守卫:每次 HTTP 尝试前取限流名额,尝试结束后记录统计。
// BeforeAttempt 返回错误时本次方法调用立即结束,不再发起后续尝试。
type CallGuard interface {
BeforeAttempt(ctx context.Context, call CallInfo) error
AfterAttempt(ctx context.Context, call CallInfo, callErr error, duration time.Duration)
}
type cardCallKey struct{}
type carrierKey struct{}
type exemptKey struct{}
type rateLimitWaitKey struct{}
type cardCall struct {
operation string
cardNo string
}
// WithCarrier 声明本次调用目标卡所属的运营商账户(本系统 tb_carrier.id),避免守卫按卡号再查一次。
func WithCarrier(ctx context.Context, carrierID uint) context.Context {
if carrierID == 0 {
return ctx
}
return context.WithValue(ctx, carrierKey{}, carrierID)
}
// WithRateLimitExempt 声明本次调用豁免运营商账户限流,只允许后台人工点击、通道阈值、自动续费、手动刷新使用。
func WithRateLimitExempt(ctx context.Context, source string) context.Context {
if source == "" {
return ctx
}
return context.WithValue(ctx, exemptKey{}, source)
}
// WithRateLimitWait 声明本次调用在拿不到限流名额时等待,总等待不超过 maxWait;超时返回限流错误。
// 只用于不占 Asynq 协程的执行方(按运营商账户调度的 NSQ 消费者);未声明时拿不到名额立即返回。
func WithRateLimitWait(ctx context.Context, maxWait time.Duration) context.Context {
if maxWait <= 0 {
return ctx
}
return context.WithValue(ctx, rateLimitWaitKey{}, maxWait)
}
// RateLimitWait 返回 ctx 中声明的限流最长等待;未声明返回 0。
func RateLimitWait(ctx context.Context) time.Duration {
wait, _ := ctx.Value(rateLimitWaitKey{}).(time.Duration)
return wait
}
// RateLimitExemptSource 返回 ctx 中声明的豁免来源,未声明返回空串。
func RateLimitExemptSource(ctx context.Context) string {
source, _ := ctx.Value(exemptKey{}).(string)
return source
}
func withCardCall(ctx context.Context, operation, cardNo string) context.Context {
return context.WithValue(ctx, cardCallKey{}, cardCall{operation: operation, cardNo: cardNo})
}
// cardCallInfo 从 ctx 取出本次卡网关调用信息;非卡操作返回 false。
func cardCallInfo(ctx context.Context) (CallInfo, bool) {
call, ok := ctx.Value(cardCallKey{}).(cardCall)
if !ok {
return CallInfo{}, false
}
carrierID, _ := ctx.Value(carrierKey{}).(uint)
return CallInfo{
Operation: call.operation, CardNo: call.cardNo, CarrierID: carrierID,
ExemptSource: RateLimitExemptSource(ctx),
}, true
}
// SetCallGuard 设置卡网关调用守卫;为 nil 时不限流、不统计。
func (c *Client) SetCallGuard(guard CallGuard) {
c.guard = guard
}

View File

@@ -11,6 +11,7 @@ import (
"net/http"
"time"
"github.com/break/junhong_cmp_fiber/pkg/constants"
"github.com/break/junhong_cmp_fiber/pkg/errors"
"github.com/bytedance/sonic"
"go.uber.org/zap"
@@ -36,6 +37,7 @@ type Client struct {
timeout time.Duration
logger *zap.Logger
maxRetries int
guard CallGuard // 卡网关调用守卫(限流与统计),为 nil 时不生效
}
type attemptObserverKey struct{}
@@ -138,6 +140,7 @@ func (c *Client) executeWithRetry(ctx context.Context, path, encryptedData strin
// 带重试的 HTTP 请求
var lastErr error
observer, _ := ctx.Value(attemptObserverKey{}).(AttemptObserver)
call, isCardCall := cardCallInfo(ctx)
for attempt := 0; attempt <= c.maxRetries; attempt++ {
if attempt > 0 {
// 检查用户 Context 是否已取消
@@ -158,12 +161,30 @@ func (c *Client) executeWithRetry(ctx context.Context, path, encryptedData strin
}
attemptNumber := attempt + 1
// 卡网关操作每次 HTTP 尝试前各取一次限流名额,内部重试同样计量;拿不到名额时本次调用立即结束。
if isCardCall && c.guard != nil {
call.Attempt = attemptNumber
if err := c.guard.BeforeAttempt(ctx, call); err != nil {
if attempt > 0 {
c.logger.Warn("Gateway 重试被限流中止", zap.String("path", path), zap.Int("attempt", attemptNumber), zap.NamedError("last_error", lastErr))
}
return nil, err
}
}
if observer != nil {
if err := observer.BeforeAttempt(ctx, attemptNumber); err != nil {
return nil, err
}
}
attemptStartedAt := time.Now()
result, retryable, err := c.executeHTTPRequest(ctx, path, encryptedData)
attemptDuration := time.Since(attemptStartedAt)
if gwErr, ok := AsGatewayError(err); ok {
gwErr.Duration = attemptDuration
}
if isCardCall && c.guard != nil {
c.guard.AfterAttempt(ctx, call, err, attemptDuration)
}
if observer != nil {
if observeErr := observer.AfterAttempt(ctx, attemptNumber, err); observeErr != nil {
return nil, observeErr
@@ -238,30 +259,32 @@ func (c *Client) executeHTTPRequest(ctx context.Context, path string, encryptedD
if err != nil {
// 用户 Context 已取消 — 不可重试
if ctx.Err() != nil {
return nil, false, errors.Wrap(errors.CodeGatewayError, ctx.Err(), "请求被取消")
return nil, false, newGatewayError(constants.GatewayErrorNetwork, errors.Wrap(errors.CodeGatewayError, ctx.Err(), "请求被取消"))
}
// Client 超时 — 可重试
if reqCtx.Err() == context.DeadlineExceeded {
return nil, true, errors.Wrap(errors.CodeGatewayTimeout, err, "Gateway 请求超时")
return nil, true, newGatewayError(constants.GatewayErrorTimeout, errors.Wrap(errors.CodeGatewayTimeout, err, "Gateway 请求超时"))
}
// 其他网络错误(连接失败、DNS 解析等)— 可重试
return nil, true, errors.Wrap(errors.CodeGatewayError, err, "发送 HTTP 请求失败")
return nil, true, newGatewayError(constants.GatewayErrorNetwork, errors.Wrap(errors.CodeGatewayError, err, "发送 HTTP 请求失败"))
}
defer resp.Body.Close()
// HTTP 状态码错误 — 不可重试
if resp.StatusCode != http.StatusOK {
return nil, false, errors.New(errors.CodeGatewayError, fmt.Sprintf("HTTP 状态码异常: %d", resp.StatusCode))
gwErr := newGatewayError(constants.GatewayErrorHTTP, errors.New(errors.CodeGatewayError, fmt.Sprintf("HTTP 状态码异常: %d", resp.StatusCode)))
gwErr.HTTPStatus = resp.StatusCode
return nil, false, gwErr
}
body, err := io.ReadAll(resp.Body)
if err != nil {
return nil, false, errors.Wrap(errors.CodeGatewayInvalidResp, err, "读取响应体失败")
return nil, false, newGatewayError(constants.GatewayErrorInvalidResponse, errors.Wrap(errors.CodeGatewayInvalidResp, err, "读取响应体失败"))
}
var gatewayResp GatewayResponse
if err := sonic.Unmarshal(body, &gatewayResp); err != nil {
return nil, false, errors.Wrap(errors.CodeGatewayInvalidResp, err, "解析 Gateway 响应失败")
return nil, false, newGatewayError(constants.GatewayErrorInvalidResponse, errors.Wrap(errors.CodeGatewayInvalidResp, err, "解析 Gateway 响应失败"))
}
// Gateway 业务错误 — 不可重试
@@ -271,7 +294,10 @@ func (c *Client) executeHTTPRequest(ctx context.Context, path string, encryptedD
zap.Int("gateway_code", gatewayResp.Code),
zap.String("gateway_msg", gatewayResp.Msg),
)
return nil, false, errors.New(errors.CodeGatewayError, fmt.Sprintf("Gateway 业务错误: code=%d, msg=%s", gatewayResp.Code, gatewayResp.Msg))
gwErr := newGatewayError(constants.GatewayErrorUpstream, errors.New(errors.CodeGatewayError, fmt.Sprintf("Gateway 业务错误: code=%d, msg=%s", gatewayResp.Code, gatewayResp.Msg)))
gwErr.ProviderCode = gatewayResp.Code
gwErr.ProviderMessage = truncateProviderMessage(gatewayResp.Msg)
return nil, false, gwErr
}
return gatewayResp.Data, false, nil

View File

@@ -4,41 +4,74 @@ package gateway
import (
"context"
"github.com/break/junhong_cmp_fiber/pkg/constants"
"github.com/break/junhong_cmp_fiber/pkg/errors"
)
// QueryCardStatus 查询流量卡状态
// POST /flow-card/status
func (c *Client) QueryCardStatus(ctx context.Context, req *CardStatusReq) (*CardStatusResp, error) {
ctx = withCardCall(ctx, constants.GatewayOpQueryStatus, cardNoOf(req))
return doRequestWithResponse[CardStatusResp](c, ctx, "/flow-card/status", req)
}
// QueryFlow 查询流量使用情况
// POST /flow-card/flow
func (c *Client) QueryFlow(ctx context.Context, req *FlowQueryReq) (*FlowUsageResp, error) {
ctx = withCardCall(ctx, constants.GatewayOpQueryFlow, flowCardNoOf(req))
return doRequestWithResponse[FlowUsageResp](c, ctx, "/flow-card/flow", req)
}
// QueryRealnameStatus 查询实名认证状态
// POST /flow-card/realName
func (c *Client) QueryRealnameStatus(ctx context.Context, req *CardStatusReq) (*RealnameStatusResp, error) {
ctx = withCardCall(ctx, constants.GatewayOpQueryRealname, cardNoOf(req))
return doRequestWithResponse[RealnameStatusResp](c, ctx, "/flow-card/realName", req)
}
// StopCard 流量卡停机
// POST /flow-card/cardStop
// 停机是外部状态写入:超时或连接失败时上游可能已受理,内部重试会对同一张卡连发并撞上游单卡频率限制,故只发一次请求。
func (c *Client) StopCard(ctx context.Context, req *CardOperationReq) error {
_, err := c.doRequest(ctx, "/flow-card/cardStop", req)
ctx = withCardCall(ctx, constants.GatewayOpStop, operationCardNoOf(req))
requestClient := *c
requestClient.maxRetries = 0
_, err := requestClient.doRequest(ctx, "/flow-card/cardStop", req)
return err
}
// StartCard 流量卡复机
// POST /flow-card/cardStart
// 复机同停机,只发一次请求,不使用内部重试。
func (c *Client) StartCard(ctx context.Context, req *CardOperationReq) error {
_, err := c.doRequest(ctx, "/flow-card/cardStart", req)
ctx = withCardCall(ctx, constants.GatewayOpStart, operationCardNoOf(req))
requestClient := *c
requestClient.maxRetries = 0
_, err := requestClient.doRequest(ctx, "/flow-card/cardStart", req)
return err
}
func cardNoOf(req *CardStatusReq) string {
if req == nil {
return ""
}
return req.CardNo
}
func flowCardNoOf(req *FlowQueryReq) string {
if req == nil {
return ""
}
return req.CardNo
}
func operationCardNoOf(req *CardOperationReq) string {
if req == nil {
return ""
}
return req.CardNo
}
// GetRealnameLink 获取实名认证跳转链接
// POST /flow-card/RealNameVerification
func (c *Client) GetRealnameLink(ctx context.Context, req *CardStatusReq) (*RealnameLinkResp, error) {

View File

@@ -11,6 +11,7 @@ import (
"github.com/gofiber/fiber/v2"
"github.com/hibiken/asynq"
"github.com/break/junhong_cmp_fiber/internal/gateway"
"github.com/break/junhong_cmp_fiber/internal/handler/validation"
dto "github.com/break/junhong_cmp_fiber/internal/model/dto"
packageExpiryQuery "github.com/break/junhong_cmp_fiber/internal/query/packageexpiry"
@@ -447,7 +448,7 @@ func (h *AssetHandler) Stop(c *fiber.Ctx) error {
switch asset.AssetType {
case "device":
result, devErr := h.deviceService.StopDevice(c.UserContext(), asset.AssetID)
result, devErr := h.deviceService.StopDevice(gateway.WithRateLimitExempt(c.UserContext(), constants.RateLimitExemptManualClick), asset.AssetID)
if devErr != nil {
return devErr
}
@@ -456,7 +457,7 @@ func (h *AssetHandler) Stop(c *fiber.Ctx) error {
if asset.ICCID == "" {
return errors.New(errors.CodeInternalError, "卡 ICCID 为空,无法停机")
}
if cardErr := h.iotCardStopResume.ManualStopCard(c.UserContext(), asset.ICCID); cardErr != nil {
if cardErr := h.iotCardStopResume.ManualStopCard(gateway.WithRateLimitExempt(c.UserContext(), constants.RateLimitExemptManualClick), asset.ICCID); cardErr != nil {
return cardErr
}
return response.Success(c, nil)
@@ -475,7 +476,7 @@ func (h *AssetHandler) Start(c *fiber.Ctx) error {
switch asset.AssetType {
case "device":
if devErr := h.deviceService.StartDevice(c.UserContext(), asset.AssetID); devErr != nil {
if devErr := h.deviceService.StartDevice(gateway.WithRateLimitExempt(c.UserContext(), constants.RateLimitExemptManualClick), asset.AssetID); devErr != nil {
return devErr
}
return response.Success(c, nil)
@@ -483,7 +484,7 @@ func (h *AssetHandler) Start(c *fiber.Ctx) error {
if asset.ICCID == "" {
return errors.New(errors.CodeInternalError, "卡 ICCID 为空,无法复机")
}
if cardErr := h.iotCardStopResume.ManualStartCard(c.UserContext(), asset.ICCID); cardErr != nil {
if cardErr := h.iotCardStopResume.ManualStartCard(gateway.WithRateLimitExempt(c.UserContext(), constants.RateLimitExemptManualClick), asset.ICCID); cardErr != nil {
return cardErr
}
return response.Success(c, nil)

View File

@@ -0,0 +1,195 @@
package admin
import (
"strings"
"github.com/go-playground/validator/v10"
"github.com/gofiber/fiber/v2"
"github.com/break/junhong_cmp_fiber/internal/model/dto"
pollingSvc "github.com/break/junhong_cmp_fiber/internal/service/polling"
"github.com/break/junhong_cmp_fiber/pkg/errors"
"github.com/break/junhong_cmp_fiber/pkg/response"
)
// CarrierSchedulerHandler 按运营商账户调度的后台接口:调度参数、请求检查场景、失败告警、单卡轮询状态、按账户监控。
type CarrierSchedulerHandler struct {
service *pollingSvc.CarrierSchedulerService
validator *validator.Validate
}
// NewCarrierSchedulerHandler 创建按运营商账户调度 Handler。
func NewCarrierSchedulerHandler(service *pollingSvc.CarrierSchedulerService, validator *validator.Validate) *CarrierSchedulerHandler {
return &CarrierSchedulerHandler{service: service, validator: validator}
}
func (h *CarrierSchedulerHandler) validate(prefix string, req any) error {
if err := h.validator.Struct(req); err != nil {
return errors.New(errors.CodeInvalidParam, validationMessage(prefix, req, err))
}
return nil
}
// ListParams 查询运营商账户调度参数列表。GET /api/admin/polling-carrier-params
func (h *CarrierSchedulerHandler) ListParams(c *fiber.Ctx) error {
result, err := h.service.ListParams(c.UserContext())
if err != nil {
return err
}
return response.Success(c, result)
}
// GetParam 查询运营商账户调度参数。GET /api/admin/polling-carrier-params/:carrier_id
func (h *CarrierSchedulerHandler) GetParam(c *fiber.Ctx) error {
carrierID, err := rateLimitPathID(c, "carrier_id", "运营商账户 ID")
if err != nil {
return err
}
result, err := h.service.GetParam(c.UserContext(), carrierID)
if err != nil {
return err
}
return response.Success(c, result)
}
// SaveParam 保存运营商账户调度参数。PUT /api/admin/polling-carrier-params/:carrier_id
func (h *CarrierSchedulerHandler) SaveParam(c *fiber.Ctx) error {
var req dto.SaveCarrierPollingParamRequest
if err := c.BodyParser(&req); err != nil {
return errors.New(errors.CodeInvalidParam, "请求参数解析失败")
}
carrierID, err := rateLimitPathID(c, "carrier_id", "运营商账户 ID")
if err != nil {
return err
}
req.CarrierID = carrierID
if err := h.validate("运营商账户调度参数不合法", &req); err != nil {
return err
}
result, err := h.service.SaveParam(c.UserContext(), &req)
if err != nil {
return err
}
return response.Success(c, result)
}
// DeleteParam 删除运营商账户调度参数(恢复全局默认)。DELETE /api/admin/polling-carrier-params/:carrier_id
func (h *CarrierSchedulerHandler) DeleteParam(c *fiber.Ctx) error {
carrierID, err := rateLimitPathID(c, "carrier_id", "运营商账户 ID")
if err != nil {
return err
}
if err := h.service.DeleteParam(c.UserContext(), carrierID); err != nil {
return err
}
return response.Success(c, nil)
}
// ListScenes 查询请求检查场景。GET /api/admin/polling-check-scenes
func (h *CarrierSchedulerHandler) ListScenes(c *fiber.Ctx) error {
result, err := h.service.ListScenes(c.UserContext())
if err != nil {
return err
}
return response.Success(c, result)
}
// UpdateScene 修改请求检查场景。PUT /api/admin/polling-check-scenes/:scene
func (h *CarrierSchedulerHandler) UpdateScene(c *fiber.Ctx) error {
var req dto.UpdatePollingCheckSceneRequest
if err := c.BodyParser(&req); err != nil {
return errors.New(errors.CodeInvalidParam, "请求参数解析失败")
}
req.Scene = strings.TrimSpace(c.Params("scene"))
if req.Scene == "" {
return errors.New(errors.CodeInvalidParam, "无效的场景编码")
}
if err := h.validate("请求检查场景参数不合法", &req); err != nil {
return err
}
result, err := h.service.UpdateScene(c.UserContext(), &req)
if err != nil {
return err
}
return response.Success(c, result)
}
// ListAlerts 查询卡轮询失败告警列表。GET /api/admin/polling-failure-alerts
func (h *CarrierSchedulerHandler) ListAlerts(c *fiber.Ctx) error {
var req dto.PollingFailureAlertListRequest
if err := c.QueryParser(&req); err != nil {
return errors.New(errors.CodeInvalidParam, "请求参数解析失败")
}
if err := h.validate("卡轮询失败告警查询参数不合法", &req); err != nil {
return err
}
result, err := h.service.ListAlerts(c.UserContext(), &req)
if err != nil {
return err
}
return response.Success(c, result)
}
// AlertSummary 未处理告警按运营商账户 × 错误类别汇总。GET /api/admin/polling-failure-alerts/summary
func (h *CarrierSchedulerHandler) AlertSummary(c *fiber.Ctx) error {
result, err := h.service.AlertSummary(c.UserContext())
if err != nil {
return err
}
return response.Success(c, result)
}
// GetAlert 查询卡轮询失败告警详情。GET /api/admin/polling-failure-alerts/:id
func (h *CarrierSchedulerHandler) GetAlert(c *fiber.Ctx) error {
id, err := rateLimitPathID(c, "id", "告警 ID")
if err != nil {
return err
}
result, err := h.service.GetAlert(c.UserContext(), id)
if err != nil {
return err
}
return response.Success(c, result)
}
// HandleAlert 标记卡轮询失败告警已处理。PUT /api/admin/polling-failure-alerts/:id/status
func (h *CarrierSchedulerHandler) HandleAlert(c *fiber.Ctx) error {
var req dto.HandlePollingFailureAlertRequest
if err := c.BodyParser(&req); err != nil {
return errors.New(errors.CodeInvalidParam, "请求参数解析失败")
}
id, err := rateLimitPathID(c, "id", "告警 ID")
if err != nil {
return err
}
req.ID = id
if err := h.validate("卡轮询失败告警处理参数不合法", &req); err != nil {
return err
}
if err := h.service.HandleAlert(c.UserContext(), &req); err != nil {
return err
}
return response.Success(c, nil)
}
// CardState 查询单卡轮询状态。GET /api/admin/iot-cards/:id/polling-state
func (h *CarrierSchedulerHandler) CardState(c *fiber.Ctx) error {
id, err := rateLimitPathID(c, "id", "卡 ID")
if err != nil {
return err
}
result, err := h.service.CardState(c.UserContext(), id)
if err != nil {
return err
}
return response.Success(c, result)
}
// CarrierStats 按运营商账户的轮询队列与执行监控。GET /api/admin/polling-stats/carrier-queues
func (h *CarrierSchedulerHandler) CarrierStats(c *fiber.Ctx) error {
result, err := h.service.CarrierStats(c.UserContext())
if err != nil {
return err
}
return response.Success(c, result)
}

View File

@@ -38,6 +38,10 @@ func (h *DeviceHandler) List(c *fiber.Ctx) error {
// ListDeviceTypes 查询设备类型选项。
// GET /api/admin/devices/types
func (h *DeviceHandler) ListDeviceTypes(c *fiber.Ctx) error {
userType := middleware.GetUserTypeFromContext(c.UserContext())
if userType != constants.UserTypeSuperAdmin && userType != constants.UserTypePlatform {
return errors.New(errors.CodeForbidden, constants.PlatformManagementForbiddenMessage)
}
result, err := h.service.ListDeviceTypes(c.UserContext())
if err != nil {
return err

View File

@@ -0,0 +1,343 @@
package admin
import (
"strconv"
"github.com/go-playground/validator/v10"
"github.com/gofiber/fiber/v2"
"github.com/redis/go-redis/v9"
"github.com/break/junhong_cmp_fiber/internal/model/dto"
pollingService "github.com/break/junhong_cmp_fiber/internal/service/polling"
"github.com/break/junhong_cmp_fiber/pkg/errors"
"github.com/break/junhong_cmp_fiber/pkg/response"
)
// GatewayRateLimitHandler 运营商账户网关限流、上游错误分类、停复机记录与调用统计 Handler。
type GatewayRateLimitHandler struct {
service *pollingService.GatewayRateLimitService
redis *redis.Client
validator *validator.Validate
}
// NewGatewayRateLimitHandler 创建网关限流 Handler。
func NewGatewayRateLimitHandler(service *pollingService.GatewayRateLimitService, redisClient *redis.Client, validate *validator.Validate) *GatewayRateLimitHandler {
if validate == nil {
validate = validator.New()
}
return &GatewayRateLimitHandler{service: service, redis: redisClient, validator: validate}
}
func (h *GatewayRateLimitHandler) validate(prefix string, req any) error {
if err := h.validator.Struct(req); err != nil {
return errors.New(errors.CodeInvalidParam, validationMessage(prefix, req, err))
}
return nil
}
func rateLimitPathID(c *fiber.Ctx, name, label string) (uint, error) {
value, err := strconv.ParseUint(c.Params(name), 10, 64)
if err != nil || value == 0 {
return 0, errors.New(errors.CodeInvalidParam, "无效的"+label)
}
return uint(value), nil
}
// ListGroups 查询网关限流组列表。
// GET /api/admin/polling-rate-limit-groups
func (h *GatewayRateLimitHandler) ListGroups(c *fiber.Ctx) error {
var req dto.RateLimitGroupListRequest
if err := c.QueryParser(&req); err != nil {
return errors.New(errors.CodeInvalidParam, "请求参数解析失败")
}
if err := h.validate("网关限流组查询参数不合法", &req); err != nil {
return err
}
result, err := h.service.ListGroups(c.UserContext(), &req)
if err != nil {
return err
}
return response.SuccessWithPagination(c, result.List, result.Total, result.Page, result.PageSize)
}
// GetGroup 查询网关限流组详情。
// GET /api/admin/polling-rate-limit-groups/:id
func (h *GatewayRateLimitHandler) GetGroup(c *fiber.Ctx) error {
id, err := rateLimitPathID(c, "id", "限流组 ID")
if err != nil {
return err
}
result, err := h.service.GetGroup(c.UserContext(), id)
if err != nil {
return err
}
return response.Success(c, result)
}
// CreateGroup 新建网关限流组。
// POST /api/admin/polling-rate-limit-groups
func (h *GatewayRateLimitHandler) CreateGroup(c *fiber.Ctx) error {
var req dto.SaveRateLimitGroupRequest
if err := c.BodyParser(&req); err != nil {
return errors.New(errors.CodeInvalidParam, "请求参数解析失败")
}
if err := h.validate("网关限流组参数不合法", &req); err != nil {
return err
}
result, err := h.service.CreateGroup(c.UserContext(), &req)
if err != nil {
return err
}
return response.Success(c, result)
}
// UpdateGroup 修改网关限流组。
// PUT /api/admin/polling-rate-limit-groups/:id
func (h *GatewayRateLimitHandler) UpdateGroup(c *fiber.Ctx) error {
var req dto.SaveRateLimitGroupRequest
if err := c.BodyParser(&req); err != nil {
return errors.New(errors.CodeInvalidParam, "请求参数解析失败")
}
id, err := rateLimitPathID(c, "id", "限流组 ID")
if err != nil {
return err
}
req.ID = id
if err := h.validate("网关限流组参数不合法", &req); err != nil {
return err
}
result, err := h.service.UpdateGroup(c.UserContext(), &req)
if err != nil {
return err
}
return response.Success(c, result)
}
// UpdateGroupStatus 启停网关限流组。
// PUT /api/admin/polling-rate-limit-groups/:id/status
func (h *GatewayRateLimitHandler) UpdateGroupStatus(c *fiber.Ctx) error {
var req dto.UpdateRateLimitGroupStatusRequest
if err := c.BodyParser(&req); err != nil {
return errors.New(errors.CodeInvalidParam, "请求参数解析失败")
}
id, err := rateLimitPathID(c, "id", "限流组 ID")
if err != nil {
return err
}
req.ID = id
if err := h.validate("网关限流组状态参数不合法", &req); err != nil {
return err
}
if err := h.service.UpdateGroupStatus(c.UserContext(), req.ID, *req.Status); err != nil {
return err
}
return response.Success(c, nil)
}
// DeleteGroup 删除网关限流组。
// DELETE /api/admin/polling-rate-limit-groups/:id
func (h *GatewayRateLimitHandler) DeleteGroup(c *fiber.Ctx) error {
id, err := rateLimitPathID(c, "id", "限流组 ID")
if err != nil {
return err
}
if err := h.service.DeleteGroup(c.UserContext(), id); err != nil {
return err
}
return response.Success(c, nil)
}
// ListRules 查询上游错误分类规则列表。
// GET /api/admin/gateway-error-classifications
func (h *GatewayRateLimitHandler) ListRules(c *fiber.Ctx) error {
var req dto.GatewayErrorRuleListRequest
if err := c.QueryParser(&req); err != nil {
return errors.New(errors.CodeInvalidParam, "请求参数解析失败")
}
if err := h.validate("上游错误分类规则查询参数不合法", &req); err != nil {
return err
}
result, err := h.service.ListRules(c.UserContext(), &req)
if err != nil {
return err
}
return response.SuccessWithPagination(c, result.List, result.Total, result.Page, result.PageSize)
}
// CreateRule 新建上游错误分类规则。
// POST /api/admin/gateway-error-classifications
func (h *GatewayRateLimitHandler) CreateRule(c *fiber.Ctx) error {
var req dto.SaveGatewayErrorRuleRequest
if err := c.BodyParser(&req); err != nil {
return errors.New(errors.CodeInvalidParam, "请求参数解析失败")
}
if err := h.validate("上游错误分类规则参数不合法", &req); err != nil {
return err
}
result, err := h.service.CreateRule(c.UserContext(), &req)
if err != nil {
return err
}
return response.Success(c, result)
}
// UpdateRule 修改上游错误分类规则。
// PUT /api/admin/gateway-error-classifications/:id
func (h *GatewayRateLimitHandler) UpdateRule(c *fiber.Ctx) error {
var req dto.SaveGatewayErrorRuleRequest
if err := c.BodyParser(&req); err != nil {
return errors.New(errors.CodeInvalidParam, "请求参数解析失败")
}
id, err := rateLimitPathID(c, "id", "规则 ID")
if err != nil {
return err
}
req.ID = id
if err := h.validate("上游错误分类规则参数不合法", &req); err != nil {
return err
}
result, err := h.service.UpdateRule(c.UserContext(), &req)
if err != nil {
return err
}
return response.Success(c, result)
}
// UpdateRuleStatus 启停上游错误分类规则。
// PUT /api/admin/gateway-error-classifications/:id/status
func (h *GatewayRateLimitHandler) UpdateRuleStatus(c *fiber.Ctx) error {
var req dto.UpdateGatewayErrorRuleStatusRequest
if err := c.BodyParser(&req); err != nil {
return errors.New(errors.CodeInvalidParam, "请求参数解析失败")
}
id, err := rateLimitPathID(c, "id", "规则 ID")
if err != nil {
return err
}
req.ID = id
if err := h.validate("上游错误分类规则状态参数不合法", &req); err != nil {
return err
}
if err := h.service.UpdateRuleStatus(c.UserContext(), req.ID, *req.Status); err != nil {
return err
}
return response.Success(c, nil)
}
// DeleteRule 删除上游错误分类规则。
// DELETE /api/admin/gateway-error-classifications/:id
func (h *GatewayRateLimitHandler) DeleteRule(c *fiber.Ctx) error {
id, err := rateLimitPathID(c, "id", "规则 ID")
if err != nil {
return err
}
if err := h.service.DeleteRule(c.UserContext(), id); err != nil {
return err
}
return response.Success(c, nil)
}
// TestRule 用已启用规则(可附带草稿)对一段上游返回信息分类,不写任何数据。
// POST /api/admin/gateway-error-classifications/test
func (h *GatewayRateLimitHandler) TestRule(c *fiber.Ctx) error {
var req dto.TestGatewayErrorRuleRequest
if err := c.BodyParser(&req); err != nil {
return errors.New(errors.CodeInvalidParam, "请求参数解析失败")
}
if err := h.validate("上游错误分类测试参数不合法", &req); err != nil {
return err
}
result, err := h.service.TestRule(c.UserContext(), &req)
if err != nil {
return err
}
return response.Success(c, result)
}
// ListStopResumeRecords 查询停复机记录。
// GET /api/admin/card-stop-resume-records
func (h *GatewayRateLimitHandler) ListStopResumeRecords(c *fiber.Ctx) error {
var req dto.StopResumeRecordListRequest
if err := c.QueryParser(&req); err != nil {
return errors.New(errors.CodeInvalidParam, "请求参数解析失败,时间请使用 RFC3339 格式")
}
if err := h.validate("停复机记录查询参数不合法", &req); err != nil {
return err
}
result, err := h.service.ListStopResumeRecords(c.UserContext(), &req)
if err != nil {
return err
}
return response.SuccessWithPagination(c, result.List, result.Total, result.Page, result.PageSize)
}
// ListCardStopResumeRecords 查询单卡停复机历史。
// GET /api/admin/iot-cards/:id/stop-resume-records
func (h *GatewayRateLimitHandler) ListCardStopResumeRecords(c *fiber.Ctx) error {
var req dto.CardStopResumeRecordListRequest
if err := c.QueryParser(&req); err != nil {
return errors.New(errors.CodeInvalidParam, "请求参数解析失败,时间请使用 RFC3339 格式")
}
id, err := rateLimitPathID(c, "id", "IoT 卡 ID")
if err != nil {
return err
}
req.ID = id
if err := h.validate("单卡停复机记录查询参数不合法", &req); err != nil {
return err
}
result, err := h.service.ListCardStopResumeRecords(c.UserContext(), &req)
if err != nil {
return err
}
return response.SuccessWithPagination(c, result.List, result.Total, result.Page, result.PageSize)
}
// CarrierCallStats 按运营商账户汇总最近若干分钟的网关调用。
// GET /api/admin/polling-stats/carriers
func (h *GatewayRateLimitHandler) CarrierCallStats(c *fiber.Ctx) error {
var req dto.GatewayCallStatsRequest
if err := c.QueryParser(&req); err != nil {
return errors.New(errors.CodeInvalidParam, "请求参数解析失败")
}
if err := h.validate("网关调用统计参数不合法", &req); err != nil {
return err
}
result, err := h.service.CarrierCallStats(c.UserContext(), h.redis, &req)
if err != nil {
return err
}
return response.Success(c, result)
}
// CarrierCallTrend 查询单个运营商账户的网关调用趋势。
// GET /api/admin/polling-stats/carriers/:carrier_id/trend
func (h *GatewayRateLimitHandler) CarrierCallTrend(c *fiber.Ctx) error {
var req dto.GatewayCallTrendRequest
if err := c.QueryParser(&req); err != nil {
return errors.New(errors.CodeInvalidParam, "请求参数解析失败")
}
carrierID, err := strconv.ParseUint(c.Params("carrier_id"), 10, 64)
if err != nil {
return errors.New(errors.CodeInvalidParam, "无效的运营商账户 ID")
}
req.CarrierID = uint(carrierID)
if err := h.validate("网关调用趋势参数不合法", &req); err != nil {
return err
}
result, err := h.service.CarrierCallTrend(c.UserContext(), h.redis, &req)
if err != nil {
return err
}
return response.Success(c, result)
}
// PollingOptions 返回轮询与网关限流相关枚举。
// GET /api/admin/polling-options
func (h *GatewayRateLimitHandler) PollingOptions(c *fiber.Ctx) error {
result, err := h.service.PollingOptions(c.UserContext())
if err != nil {
return err
}
return response.Success(c, result)
}

View File

@@ -191,3 +191,98 @@ func (h *PollingConfigHandler) ListEnabled(c *fiber.Ctx) error {
return response.Success(c, configs)
}
// UpdatePriorities 按给定顺序整体调整轮询配置优先级(仅超级管理员)。
// PUT /api/admin/polling-configs/priorities
func (h *PollingConfigHandler) UpdatePriorities(c *fiber.Ctx) error {
var req dto.UpdatePollingConfigPrioritiesRequest
if err := c.BodyParser(&req); err != nil {
return errors.New(errors.CodeInvalidParam, "请求参数解析失败")
}
if len(req.OrderedIDs) == 0 {
return errors.New(errors.CodeInvalidParam, "轮询配置优先级参数不合法:配置 ID 列表不能为空")
}
if err := h.service.UpdatePriorities(c.UserContext(), &req); err != nil {
return err
}
return response.Success(c, nil)
}
// MatchPreview 预览一张卡的轮询配置匹配结果;GET 只按当前配置,POST 可带未保存修改,均不写数据。
// GET /api/admin/polling-configs/match-preview
// POST /api/admin/polling-configs/match-preview
func (h *PollingConfigHandler) MatchPreview(c *fiber.Ctx) error {
var req dto.PollingConfigMatchPreviewRequest
if c.Method() == fiber.MethodPost {
if err := c.BodyParser(&req); err != nil {
return errors.New(errors.CodeInvalidParam, "请求参数解析失败")
}
} else {
var query dto.PollingConfigMatchPreviewQuery
if err := c.QueryParser(&query); err != nil {
return errors.New(errors.CodeInvalidParam, "请求参数解析失败")
}
req.ICCID = query.ICCID
}
if len(req.Overrides) > 20 {
return errors.New(errors.CodeInvalidParam, "匹配预览参数不合法:未保存修改最多 20 条")
}
if req.ICCID == "" {
return errors.New(errors.CodeInvalidParam, "匹配预览参数不合法:ICCID 不能为空")
}
result, err := h.service.MatchPreview(c.UserContext(), &req)
if err != nil {
return err
}
return response.Success(c, result)
}
// CreateRecalculateTask 创建"立即重算到期时间"异步任务(仅超级管理员)。
// POST /api/admin/polling-configs/recalculate-due
func (h *PollingConfigHandler) CreateRecalculateTask(c *fiber.Ctx) error {
var req dto.PollingRecalculateDueRequest
if len(c.Body()) > 0 {
if err := c.BodyParser(&req); err != nil {
return errors.New(errors.CodeInvalidParam, "请求参数解析失败")
}
}
result, err := h.service.CreateRecalculateTask(c.UserContext(), &req)
if err != nil {
return err
}
return response.Success(c, result)
}
// GetRecalculateTask 查询重算到期时间任务进度。
// GET /api/admin/polling-configs/recalculate-due/:task_id
func (h *PollingConfigHandler) GetRecalculateTask(c *fiber.Ctx) error {
taskID := c.Params("task_id")
if taskID == "" {
return errors.New(errors.CodeInvalidParam, "重算任务 ID 不能为空")
}
result, err := h.service.GetRecalculateTask(c.UserContext(), taskID)
if err != nil {
return err
}
return response.Success(c, result)
}
// TierStats 查询分层轮询统计(结果缓存 5 分钟)。
// GET /api/admin/polling-stats/tiers
func (h *PollingConfigHandler) TierStats(c *fiber.Ctx) error {
result, err := h.service.TierStats(c.UserContext())
if err != nil {
return err
}
return response.Success(c, result)
}
// PreResetStats 查询重置日前加密效果统计。
// GET /api/admin/polling-stats/pre-reset
func (h *PollingConfigHandler) PreResetStats(c *fiber.Ctx) error {
result, err := h.service.PreResetStats(c.UserContext())
if err != nil {
return err
}
return response.Success(c, result)
}

View File

@@ -157,7 +157,6 @@ func (h *PollingManualTriggerHandler) GetStatus(c *fiber.Ctx) error {
constants.TaskTypePollingRealname,
constants.TaskTypePollingCarddata,
constants.TaskTypePollingPackage,
constants.TaskTypePollingProtect,
} {
size, _ := h.service.GetQueueSize(ctx, taskType)
queueSizes[taskType] = size
@@ -272,6 +271,7 @@ func (h *PollingManualTriggerHandler) getTaskTypeName(taskType string) string {
case constants.TaskTypePollingPackage:
return "套餐检查"
case constants.TaskTypePollingProtect:
// 保护期已下线,保留名称供历史手动触发记录展示。
return "保护期检查"
default:
return taskType

View File

@@ -0,0 +1,81 @@
package admin
import (
"github.com/go-playground/validator/v10"
"github.com/gofiber/fiber/v2"
"github.com/break/junhong_cmp_fiber/internal/handler/validation"
"github.com/break/junhong_cmp_fiber/internal/model/dto"
"github.com/break/junhong_cmp_fiber/internal/query/stopresumerule"
exportTaskService "github.com/break/junhong_cmp_fiber/internal/service/export_task"
"github.com/break/junhong_cmp_fiber/pkg/constants"
"github.com/break/junhong_cmp_fiber/pkg/errors"
"github.com/break/junhong_cmp_fiber/pkg/middleware"
"github.com/break/junhong_cmp_fiber/pkg/response"
)
// StopResumeRuleHandler 停复机规则演练 Handler:只判断不执行,仅超级管理员与平台账号可用。
type StopResumeRuleHandler struct {
query *stopresumerule.Query
exportService *exportTaskService.Service
validator *validator.Validate
}
// NewStopResumeRuleHandler 创建停复机规则演练 Handler。
func NewStopResumeRuleHandler(query *stopresumerule.Query, exportService *exportTaskService.Service, validator *validator.Validate) *StopResumeRuleHandler {
return &StopResumeRuleHandler{query: query, exportService: exportService, validator: validator}
}
// DryRun 执行停复机规则演练。
// GET /api/admin/stop-resume-rule/dry-run
func (h *StopResumeRuleHandler) DryRun(c *fiber.Ctx) error {
var req dto.StopResumeRuleDryRunRequest
if err := c.QueryParser(&req); err != nil {
return errors.New(errors.CodeInvalidParam, "请求参数格式不正确")
}
if err := h.validator.Struct(&req); err != nil {
return errors.New(errors.CodeInvalidParam, validation.Message("停复机规则演练参数不正确", &req, err))
}
result, err := h.query.DryRun(c.UserContext(), &req)
if err != nil {
return err
}
return response.Success(c, result)
}
// Export 创建停复机规则演练明细异步导出任务。
// POST /api/admin/stop-resume-rule/dry-run/export
func (h *StopResumeRuleHandler) Export(c *fiber.Ctx) error {
if err := stopresumerule.EnsureAllowed(middleware.GetUserTypeFromContext(c.UserContext())); err != nil {
return err
}
var req dto.StopResumeRuleDryRunExportRequest
if err := c.BodyParser(&req); err != nil {
return errors.New(errors.CodeInvalidParam, "请求参数格式不正确")
}
if err := h.validator.Struct(&req); err != nil {
return errors.New(errors.CodeInvalidParam, validation.Message("停复机规则演练导出参数不正确", &req, err))
}
// 创建期校验筛选,避免任务执行时才因参数非法失败。
if _, err := stopresumerule.BuildFilter(req.CarrierID, req.StopReason, req.AssumeControlOn, req.AssumeControlOff); err != nil {
return err
}
format := req.Format
if format == "" {
format = constants.ExportTaskFormatXLSX
}
filters := map[string]interface{}{
"stop_reason": req.StopReason, "assume_control_on": req.AssumeControlOn, "assume_control_off": req.AssumeControlOff,
}
if req.CarrierID != nil {
filters["carrier_id"] = *req.CarrierID
}
result, err := h.exportService.CreateTask(c.UserContext(), &dto.CreateExportTaskRequest{
Scene: constants.ExportTaskSceneStopResumeRuleDryRun, Format: format,
Query: map[string]interface{}{"filters": filters},
})
if err != nil {
return err
}
return response.Success(c, result)
}

View File

@@ -5,7 +5,9 @@ import (
"context"
assetAutoRenewalApp "github.com/break/junhong_cmp_fiber/internal/application/assetautorenewal"
"github.com/break/junhong_cmp_fiber/internal/gateway"
iotCardSvc "github.com/break/junhong_cmp_fiber/internal/service/iot_card"
"github.com/break/junhong_cmp_fiber/pkg/constants"
"github.com/break/junhong_cmp_fiber/pkg/errors"
)
@@ -35,7 +37,7 @@ func (c *CardCommander) ResumeAssetForAutoRenewal(ctx context.Context, assetType
if c == nil || c.service == nil {
return assetAutoRenewalApp.ResumeOutcome{}, errCommanderUnavailable()
}
outcome, err := c.service.ResumeAssetForAutoRenewal(ctx, assetType, assetID)
outcome, err := c.service.ResumeAssetForAutoRenewal(gateway.WithRateLimitExempt(ctx, constants.RateLimitExemptAutoRenewal), assetType, assetID)
return assetAutoRenewalApp.ResumeOutcome{
Applied: outcome.Applied, IntegrationID: outcome.IntegrationID,
Result: outcome.Result, SafeReason: outcome.SafeReason,
@@ -47,7 +49,7 @@ func (c *CardCommander) QueryAutoRenewalResumeState(ctx context.Context, assetTy
if c == nil || c.service == nil {
return false, false, "", errCommanderUnavailable()
}
return c.service.QueryAutoRenewalResumeState(ctx, assetType, assetID)
return c.service.QueryAutoRenewalResumeState(gateway.WithRateLimitExempt(ctx, constants.RateLimitExemptAutoRenewal), assetType, assetID)
}
// errCommanderUnavailable 返回复机执行端口未配置的稳定错误。

View File

@@ -87,6 +87,17 @@ func NewRegistry() *Registry {
iotCardDeactivated := iotCardAction(constants.AuditActionIotCardDeactivated, "停用 IoT 卡资产", constants.AuditActorAccount, constants.AuditSourceAdminAPI)
iotCardDeactivated.Risk = constants.AuditRiskHigh
iotCardPollingStatusUpdated := iotCardAction(constants.AuditActionIotCardPollingStatusUpdated, "更新 IoT 卡轮询开关", constants.AuditActorAccount, constants.AuditSourceAdminAPI)
iotCardTerminalPollingDisabled := iotCardAction(constants.AuditActionIotCardTerminalPollingDisabled, "上游终态自动关闭 IoT 卡轮询", constants.AuditActorSystemTask, constants.AuditSourceWorker)
iotCardTerminalPollingDisabled.AllowedActor, iotCardTerminalPollingDisabled.Source = "", ""
// 停复机可能由人工、Worker、定时任务、开放接口、C 端与回调触发,命中终态后均需留下关闭轮询的审计事实。
iotCardTerminalPollingDisabled.AllowedOrigins = []ActionOrigin{
{Actor: constants.AuditActorAccount, Source: constants.AuditSourceAdminAPI},
{Actor: constants.AuditActorSystemTask, Source: constants.AuditSourceWorker},
{Actor: constants.AuditActorScheduledJob, Source: constants.AuditSourceScheduler},
{Actor: constants.AuditActorOpenAPI, Source: constants.AuditSourceOpenAPI},
{Actor: constants.AuditActorPersonalCustomer, Source: constants.AuditSourcePersonalAPI},
{Actor: constants.AuditActorExternalSystem, Source: constants.AuditSourceCallback},
}
iotCardPollingStatusBatchUpdated := iotCardBatchAction(constants.AuditActionIotCardPollingStatusBatchUpdated, "批量更新 IoT 卡轮询开关")
iotCardBatchDeleted := ActionDefinition{
Code: constants.AuditActionIotCardBatchDeleted, Name: "批量删除 IoT 卡",
@@ -259,6 +270,18 @@ func NewRegistry() *Registry {
pollingConfigUpdated := pollingAction(constants.AuditActionPollingConfigUpdated, "更新轮询配置", constants.AuditResourcePollingConfig, constants.AuditRiskHigh)
pollingConfigDeleted := pollingAction(constants.AuditActionPollingConfigDeleted, "删除轮询配置", constants.AuditResourcePollingConfig, constants.AuditRiskHigh)
pollingConfigStatusUpdated := pollingAction(constants.AuditActionPollingConfigStatusUpdated, "更新轮询配置状态", constants.AuditResourcePollingConfig, constants.AuditRiskHigh)
rateLimitGroupCreated := pollingAction(constants.AuditActionRateLimitGroupCreated, "创建网关限流组", constants.AuditResourceRateLimitGroup, constants.AuditRiskHigh)
rateLimitGroupUpdated := pollingAction(constants.AuditActionRateLimitGroupUpdated, "更新网关限流组", constants.AuditResourceRateLimitGroup, constants.AuditRiskHigh)
rateLimitGroupStatusUpdated := pollingAction(constants.AuditActionRateLimitGroupStatusUpdated, "更新网关限流组状态", constants.AuditResourceRateLimitGroup, constants.AuditRiskHigh)
rateLimitGroupDeleted := pollingAction(constants.AuditActionRateLimitGroupDeleted, "删除网关限流组", constants.AuditResourceRateLimitGroup, constants.AuditRiskHigh)
gatewayErrorRuleCreated := pollingAction(constants.AuditActionGatewayErrorRuleCreated, "创建上游错误分类规则", constants.AuditResourceGatewayErrorRule, constants.AuditRiskNormal)
gatewayErrorRuleUpdated := pollingAction(constants.AuditActionGatewayErrorRuleUpdated, "更新上游错误分类规则", constants.AuditResourceGatewayErrorRule, constants.AuditRiskNormal)
gatewayErrorRuleStatusUpdated := pollingAction(constants.AuditActionGatewayErrorRuleStatusUpdated, "更新上游错误分类规则状态", constants.AuditResourceGatewayErrorRule, constants.AuditRiskNormal)
carrierPollingParamSaved := pollingAction(constants.AuditActionCarrierPollingParamSaved, "保存运营商账户调度参数", constants.AuditResourceCarrierPollingParam, constants.AuditRiskHigh)
carrierPollingParamDeleted := pollingAction(constants.AuditActionCarrierPollingParamDeleted, "删除运营商账户调度参数", constants.AuditResourceCarrierPollingParam, constants.AuditRiskHigh)
pollingCheckSceneUpdated := pollingAction(constants.AuditActionPollingCheckSceneUpdated, "修改请求检查场景", constants.AuditResourcePollingCheckScene, constants.AuditRiskNormal)
pollingFailureAlertHandled := pollingAction(constants.AuditActionPollingFailureAlertHandled, "处理卡轮询失败告警", constants.AuditResourcePollingFailureAlert, constants.AuditRiskNormal)
gatewayErrorRuleDeleted := pollingAction(constants.AuditActionGatewayErrorRuleDeleted, "删除上游错误分类规则", constants.AuditResourceGatewayErrorRule, constants.AuditRiskNormal)
pollingConcurrencyUpdated := pollingAction(constants.AuditActionPollingConcurrencyUpdated, "更新轮询并发配置", constants.AuditResourcePollingConcurrencyConfig, constants.AuditRiskNormal)
pollingConcurrencyReset := pollingAction(constants.AuditActionPollingConcurrencyReset, "重置轮询并发计数", constants.AuditResourcePollingConcurrencyConfig, constants.AuditRiskNormal)
pollingAlertRuleCreated := pollingAction(constants.AuditActionPollingAlertRuleCreated, "创建轮询告警规则", constants.AuditResourcePollingAlertRule, constants.AuditRiskNormal)
@@ -519,6 +542,7 @@ func NewRegistry() *Registry {
constants.AuditActionIotCardDeleted: iotCardDeleted,
constants.AuditActionIotCardDeactivated: iotCardDeactivated,
constants.AuditActionIotCardPollingStatusUpdated: iotCardPollingStatusUpdated,
constants.AuditActionIotCardTerminalPollingDisabled: iotCardTerminalPollingDisabled,
constants.AuditActionIotCardPollingStatusBatchUpdated: iotCardPollingStatusBatchUpdated,
constants.AuditActionIotCardBatchDeleted: iotCardBatchDeleted,
constants.AuditActionIotCardAllocationBatch: iotCardAllocationBatch,
@@ -650,6 +674,18 @@ func NewRegistry() *Registry {
constants.AuditActionPollingConfigUpdated: pollingConfigUpdated,
constants.AuditActionPollingConfigDeleted: pollingConfigDeleted,
constants.AuditActionPollingConfigStatusUpdated: pollingConfigStatusUpdated,
constants.AuditActionRateLimitGroupCreated: rateLimitGroupCreated,
constants.AuditActionRateLimitGroupUpdated: rateLimitGroupUpdated,
constants.AuditActionRateLimitGroupStatusUpdated: rateLimitGroupStatusUpdated,
constants.AuditActionRateLimitGroupDeleted: rateLimitGroupDeleted,
constants.AuditActionGatewayErrorRuleCreated: gatewayErrorRuleCreated,
constants.AuditActionGatewayErrorRuleUpdated: gatewayErrorRuleUpdated,
constants.AuditActionGatewayErrorRuleStatusUpdated: gatewayErrorRuleStatusUpdated,
constants.AuditActionGatewayErrorRuleDeleted: gatewayErrorRuleDeleted,
constants.AuditActionCarrierPollingParamSaved: carrierPollingParamSaved,
constants.AuditActionCarrierPollingParamDeleted: carrierPollingParamDeleted,
constants.AuditActionPollingCheckSceneUpdated: pollingCheckSceneUpdated,
constants.AuditActionPollingFailureAlertHandled: pollingFailureAlertHandled,
constants.AuditActionPollingConcurrencyUpdated: pollingConcurrencyUpdated,
constants.AuditActionPollingConcurrencyReset: pollingConcurrencyReset,
constants.AuditActionPollingAlertRuleCreated: pollingAlertRuleCreated,
@@ -930,6 +966,26 @@ func NewRegistry() *Registry {
Type: constants.AuditResourcePollingConcurrencyConfig, Name: "轮询并发配置",
IdentityFields: []string{"id", "task_type", "max_concurrency"},
},
constants.AuditResourceRateLimitGroup: {
Type: constants.AuditResourceRateLimitGroup, Name: "网关限流组",
IdentityFields: []string{"id", "carrier_id", "group_code", "group_name", "max_qps", "reserved_qps", "status"},
},
constants.AuditResourceGatewayErrorRule: {
Type: constants.AuditResourceGatewayErrorRule, Name: "上游错误分类规则",
IdentityFields: []string{"id", "keyword", "match_type", "category", "priority", "status"},
},
constants.AuditResourceCarrierPollingParam: {
Type: constants.AuditResourceCarrierPollingParam, Name: "运营商账户调度参数",
IdentityFields: []string{"carrier_id", "inflight_limit", "card_max_in_flight", "priority_max_ratio"},
},
constants.AuditResourcePollingCheckScene: {
Type: constants.AuditResourcePollingCheckScene, Name: "请求检查场景",
IdentityFields: []string{"scene", "scene_name"},
},
constants.AuditResourcePollingFailureAlert: {
Type: constants.AuditResourcePollingFailureAlert, Name: "卡轮询失败告警",
IdentityFields: []string{"id", "alert_level", "card_id", "carrier_id", "check_type", "status"},
},
constants.AuditResourcePollingAlertRule: {
Type: constants.AuditResourcePollingAlertRule, Name: "轮询告警规则",
IdentityFields: []string{"id", "rule_name", "task_type", "metric_type", "operator", "threshold", "alert_level", "status"},

View File

@@ -0,0 +1,137 @@
package cardobservation
import (
"context"
"time"
"go.uber.org/zap"
"gorm.io/gorm"
cardapp "github.com/break/junhong_cmp_fiber/internal/application/cardobservation"
carddomain "github.com/break/junhong_cmp_fiber/internal/domain/cardobservation"
"github.com/break/junhong_cmp_fiber/internal/gateway"
"github.com/break/junhong_cmp_fiber/internal/model"
"github.com/break/junhong_cmp_fiber/internal/store/postgres"
"github.com/break/junhong_cmp_fiber/pkg/constants"
)
// PreActivationSettler 在资产从"无生效主套餐"变为"有生效主套餐"之前,对已切换日快照计量的卡查一次流量并按现有观测入账。
// 此时资产无生效套餐,增量只记为已结算,不扣新套餐、不产生流量扣减事件,因此也不会触发停复机评估。
// 结算尽力而为:查询失败、限流、超时只告警,不阻塞激活。
type PreActivationSettler struct {
db *gorm.DB
gateway *gateway.Client
observation *cardapp.Service
carrier *postgres.CarrierStore
logger *zap.Logger
}
// NewPreActivationSettler 创建激活前流量结算器。
func NewPreActivationSettler(db *gorm.DB, gatewayClient *gateway.Client, observation *cardapp.Service, carrier *postgres.CarrierStore, logger *zap.Logger) *PreActivationSettler {
if logger == nil {
logger = zap.NewNop()
}
return &PreActivationSettler{db: db, gateway: gatewayClient, observation: observation, carrier: carrier, logger: logger}
}
// SettleBeforeActivation 对载体(卡或设备)执行激活前流量结算;资产已有生效主套餐时直接返回。
func (s *PreActivationSettler) SettleBeforeActivation(ctx context.Context, carrierType string, carrierID uint) {
if s == nil || s.db == nil || s.gateway == nil || s.observation == nil {
return
}
hasMain, err := s.hasActiveMainPackage(ctx, carrierType, carrierID)
if err != nil {
s.logger.Warn("激活前流量结算:查询生效主套餐失败,跳过结算", zap.String("carrier_type", carrierType), zap.Uint("carrier_id", carrierID), zap.Error(err))
return
}
if hasMain {
return
}
s.logger.Info("激活前流量结算开始", zap.String("carrier_type", carrierType), zap.Uint("asset_id", carrierID))
cards, err := s.cardsOf(ctx, carrierType, carrierID)
if err != nil {
s.logger.Warn("激活前流量结算:查询载体下卡失败,跳过结算", zap.String("carrier_type", carrierType), zap.Uint("carrier_id", carrierID), zap.Error(err))
return
}
deadline := time.Now().Add(constants.PreActivationSettlementTotalTimeoutSeconds * time.Second)
for _, card := range cards {
if !s.observation.TrafficSwitched(card.ID) {
continue
}
if time.Now().After(deadline) {
s.alert(card, carrierType, carrierID, "整次结算达到总超时,跳过该卡")
continue
}
s.settleCard(ctx, card, carrierType, carrierID, deadline)
}
}
func (s *PreActivationSettler) settleCard(ctx context.Context, card *model.IotCard, carrierType string, carrierID uint, deadline time.Time) {
stepDeadline := time.Now().Add(constants.PreActivationSettlementStepTimeoutSeconds * time.Second)
if stepDeadline.After(deadline) {
stepDeadline = deadline
}
stepCtx, cancel := context.WithDeadline(context.WithoutCancel(ctx), stepDeadline)
defer cancel()
response, err := s.gateway.QueryFlow(gateway.WithCarrier(stepCtx, card.CarrierID), &gateway.FlowQueryReq{CardNo: card.ICCID})
if err != nil {
reason := "查询流量失败"
if gateway.IsRateLimited(err) {
reason = "运营商账户网关限流"
}
s.alert(card, carrierType, carrierID, reason)
return
}
resetDay := 1
if s.carrier != nil {
resetDay = s.carrier.GetDataResetDay(stepCtx, card.CarrierID)
}
observedAt := time.Now().UTC()
decision, err := s.observation.ApplyTrafficObservation(stepCtx, carddomain.TrafficObservation{
CardID: card.ID, GatewayReadingMB: float64(response.Used), ResetDay: resetDay,
Metadata: carddomain.ObservationMetadata{
ObservationID: "pre-activation-" + card.ICCID + "-" + observedAt.Format("20060102150405.000"),
Source: constants.CardObservationSourceBusinessEvent, Scene: constants.CardObservationScenePreActivationSettlement,
ObservedAt: observedAt,
},
})
if err != nil {
s.alert(card, carrierType, carrierID, "流量结算入账失败")
return
}
s.logger.Info("激活前流量结算完成", zap.Uint("card_id", card.ID), zap.String("iccid", card.ICCID),
zap.String("carrier_type", carrierType), zap.Uint("asset_id", carrierID),
zap.Float64("gateway_flow_mb", float64(response.Used)), zap.Float64("increment_mb", decision.IncrementMB),
zap.String("metric", "package.pre_activation_settlement.done"))
}
// alert 写激活前结算失败告警:该卡本月可能多计无套餐期间用量,由运营核对。
func (s *PreActivationSettler) alert(card *model.IotCard, carrierType string, carrierID uint, reason string) {
s.logger.Error("激活前流量结算未完成,该卡本月可能多计无套餐期间用量,请运营核对",
zap.Uint("card_id", card.ID), zap.String("iccid", card.ICCID), zap.String("carrier_type", carrierType),
zap.Uint("asset_id", carrierID), zap.String("reason", reason), zap.String("metric", "package.pre_activation_settlement.failed"))
}
func (s *PreActivationSettler) hasActiveMainPackage(ctx context.Context, carrierType string, carrierID uint) (bool, error) {
column := "iot_card_id"
if carrierType == constants.AssetTypeDevice {
column = "device_id"
}
var count int64
err := s.db.WithContext(ctx).Model(&model.PackageUsage{}).
Where(column+" = ? AND status = ? AND master_usage_id IS NULL", carrierID, constants.PackageUsageStatusActive).
Count(&count).Error
return count > 0, err
}
func (s *PreActivationSettler) cardsOf(ctx context.Context, carrierType string, carrierID uint) ([]*model.IotCard, error) {
var cards []*model.IotCard
if carrierType == constants.AssetTypeDevice {
err := s.db.WithContext(ctx).Model(&model.IotCard{}).
Joins("JOIN tb_device_sim_binding b ON b.iot_card_id = tb_iot_card.id AND b.bind_status = 1 AND b.deleted_at IS NULL").
Where("b.device_id = ?", carrierID).Find(&cards).Error
return cards, err
}
err := s.db.WithContext(ctx).Where("id = ?", carrierID).Find(&cards).Error
return cards, err
}

View File

@@ -9,7 +9,9 @@ import (
carrierthresholdapp "github.com/break/junhong_cmp_fiber/internal/application/carrierthreshold"
carrierthresholddomain "github.com/break/junhong_cmp_fiber/internal/domain/carrierthreshold"
"github.com/break/junhong_cmp_fiber/internal/gateway"
iotcardsvc "github.com/break/junhong_cmp_fiber/internal/service/iot_card"
"github.com/break/junhong_cmp_fiber/pkg/constants"
"github.com/break/junhong_cmp_fiber/pkg/errors"
)
@@ -34,7 +36,7 @@ func (c *CardCommander) StopCardForThreshold(ctx context.Context, cardID uint) (
if c == nil || c.service == nil {
return carrierthresholddomain.CommandOutcome{}, errors.New(errors.CodeServiceUnavailable, "通道流量阈值停机执行能力未配置")
}
return c.service.StopCardForChannelThreshold(ctx, cardID)
return c.service.StopCardForChannelThreshold(gateway.WithRateLimitExempt(ctx, constants.RateLimitExemptChannelThreshold), cardID)
}
// ResumeCardForThreshold 执行通道阈值新周期复机。
@@ -42,7 +44,7 @@ func (c *CardCommander) ResumeCardForThreshold(ctx context.Context, cardID uint)
if c == nil || c.service == nil {
return carrierthresholddomain.CommandOutcome{}, errors.New(errors.CodeServiceUnavailable, "通道流量阈值复机执行能力未配置")
}
return c.service.ResumeCardForChannelThreshold(ctx, cardID)
return c.service.ResumeCardForChannelThreshold(gateway.WithRateLimitExempt(ctx, constants.RateLimitExemptChannelThreshold), cardID)
}
// ResumeReady 判断解锁后的卡是否满足自动复机条件。
@@ -58,7 +60,7 @@ func (c *CardCommander) CardNetworkStatus(ctx context.Context, cardID uint) (int
if c == nil || c.service == nil {
return 0, false, "", errors.New(errors.CodeServiceUnavailable, "通道流量阈值状态查询能力未配置")
}
return c.service.QueryChannelThresholdCardStatus(ctx, cardID)
return c.service.QueryChannelThresholdCardStatus(gateway.WithRateLimitExempt(ctx, constants.RateLimitExemptChannelThreshold), cardID)
}
// ConfirmCardState 按已确认的运营商结果补写卡状态。

View File

@@ -0,0 +1,142 @@
package gatewaylimit
import (
"context"
"regexp"
"strings"
"sync"
"time"
"go.uber.org/zap"
"github.com/break/junhong_cmp_fiber/internal/model"
"github.com/break/junhong_cmp_fiber/pkg/constants"
)
// ClassificationLoader 读取全部启用的错误分类规则(按优先级排序)。
type ClassificationLoader interface {
ListEnabled(ctx context.Context) ([]model.GatewayErrorClassification, error)
}
// Classification 是一次上游错误的分类结果。
type Classification struct {
Category string // 见 constants.GatewayClass*
RuleID uint // 命中的规则 ID,未命中为 0
Keyword string // 命中的关键字
MaybeAccepted bool // 上游可能已受理
// AutoDisablePolling 命中规则标记自动关闭轮询:停复机不重试,关闭该卡轮询。
AutoDisablePolling bool
}
type compiledRule struct {
row model.GatewayErrorClassification
ops map[string]bool
regex *regexp.Regexp
}
// Classifier 按后台维护的关键字规则对上游错误分类;规则在进程内缓存,超过 TTL 重新加载。
type Classifier struct {
loader ClassificationLoader
logger *zap.Logger
mu sync.Mutex
rules []compiledRule
loaded time.Time
}
// NewClassifier 创建错误分类器;loader 为 nil 时一律归为业务错误。
func NewClassifier(loader ClassificationLoader, logger *zap.Logger) *Classifier {
if logger == nil {
logger = zap.NewNop()
}
return &Classifier{loader: loader, logger: logger}
}
// Invalidate 使规则缓存立即失效,后台保存规则后调用。
func (c *Classifier) Invalidate() {
if c == nil {
return
}
c.mu.Lock()
c.loaded = time.Time{}
c.mu.Unlock()
}
// Classify 对某操作的上游返回信息分类;未命中任何启用规则时归为业务错误。
func (c *Classifier) Classify(ctx context.Context, op, message string) Classification {
if c == nil {
return Classification{Category: constants.GatewayClassBusiness}
}
return MatchRules(c.current(ctx), op, message)
}
// ClassifyWith 用给定规则列表分类,供测试接口在不写数据的前提下验证规则。
func ClassifyWith(rows []model.GatewayErrorClassification, op, message string) Classification {
return MatchRules(compileRules(rows, zap.NewNop()), op, message)
}
// MatchRules 按优先级顺序返回第一个命中的规则分类。
func MatchRules(rules []compiledRule, op, message string) Classification {
for _, rule := range rules {
if len(rule.ops) > 0 && !rule.ops[op] {
continue
}
if !ruleMatches(rule, message) {
continue
}
return Classification{Category: rule.row.Category, RuleID: rule.row.ID, Keyword: rule.row.Keyword,
MaybeAccepted: rule.row.MaybeAccepted, AutoDisablePolling: rule.row.AutoDisablePolling && rule.row.Category == constants.GatewayClassTerminal}
}
return Classification{Category: constants.GatewayClassBusiness}
}
func ruleMatches(rule compiledRule, message string) bool {
switch rule.row.MatchType {
case constants.GatewayMatchPrefix:
return strings.HasPrefix(message, rule.row.Keyword)
case constants.GatewayMatchRegex:
return rule.regex != nil && rule.regex.MatchString(message)
default:
return strings.Contains(message, rule.row.Keyword)
}
}
func (c *Classifier) current(ctx context.Context) []compiledRule {
c.mu.Lock()
defer c.mu.Unlock()
if c.loader == nil || time.Since(c.loaded) < constants.GatewayRateLimitConfigTTL {
return c.rules
}
rows, err := c.loader.ListEnabled(ctx)
c.loaded = time.Now()
if err != nil {
c.logger.Warn("加载上游错误分类规则失败,沿用上一次规则", zap.Error(err))
return c.rules
}
c.rules = compileRules(rows, c.logger)
return c.rules
}
func compileRules(rows []model.GatewayErrorClassification, logger *zap.Logger) []compiledRule {
rules := make([]compiledRule, 0, len(rows))
for _, row := range rows {
rule := compiledRule{row: row, ops: map[string]bool{}}
ops, err := DecodeOperations(row.Operations)
if err != nil {
logger.Warn("错误分类规则适用操作非法,已跳过", zap.Uint("rule_id", row.ID))
continue
}
for _, op := range ops {
rule.ops[op] = true
}
if row.MatchType == constants.GatewayMatchRegex {
re, err := regexp.Compile(row.Keyword)
if err != nil {
logger.Warn("错误分类规则正则非法,已跳过", zap.Uint("rule_id", row.ID))
continue
}
rule.regex = re
}
rules = append(rules, rule)
}
return rules
}

View File

@@ -0,0 +1,130 @@
package gatewaylimit
import (
"context"
"sync"
"time"
"github.com/bytedance/sonic"
"go.uber.org/zap"
"github.com/break/junhong_cmp_fiber/internal/model"
"github.com/break/junhong_cmp_fiber/pkg/constants"
)
// defaultGroupCode 是未被任何启用限流组覆盖时使用的默认组标识(每个账户每个操作独立窗口)。
const defaultGroupCode = "default"
// GroupRule 是一个限流组对某个操作生效的规则快照。
type GroupRule struct {
GroupCode string
MaxQPS int
ReservedQPS int
HighPriority bool // 该操作在组内是否为高优先级
}
// groupIndex 是启用限流组按"账户 → 操作"建立的索引。
type groupIndex struct {
byCarrier map[uint]map[string]GroupRule
global map[string]GroupRule
}
// GroupLoader 读取全部启用的限流组。
type GroupLoader interface {
ListEnabled(ctx context.Context) ([]model.CarrierRateLimitGroup, error)
}
// GroupProvider 在进程内缓存启用限流组,超过 TTL 重新加载,后台修改数秒内生效。
type GroupProvider struct {
loader GroupLoader
logger *zap.Logger
mu sync.Mutex
index groupIndex
loaded time.Time
}
// NewGroupProvider 创建限流组提供者;loader 为 nil 时所有操作使用默认 QPS。
func NewGroupProvider(loader GroupLoader, logger *zap.Logger) *GroupProvider {
if logger == nil {
logger = zap.NewNop()
}
return &GroupProvider{loader: loader, logger: logger, index: groupIndex{byCarrier: map[uint]map[string]GroupRule{}, global: map[string]GroupRule{}}}
}
// Invalidate 使缓存立即失效,后台保存限流组后调用。
func (p *GroupProvider) Invalidate() {
if p == nil {
return
}
p.mu.Lock()
p.loaded = time.Time{}
p.mu.Unlock()
}
// Resolve 返回某账户某操作的账户组规则(未覆盖时 ok=false)与全局组规则(未配置时 ok=false)。
func (p *GroupProvider) Resolve(ctx context.Context, carrierID uint, op string) (account GroupRule, accountOK bool, global GroupRule, globalOK bool) {
index := p.current(ctx)
if rules, ok := index.byCarrier[carrierID]; ok {
account, accountOK = rules[op]
}
global, globalOK = index.global[op]
return
}
func (p *GroupProvider) current(ctx context.Context) groupIndex {
if p == nil {
return groupIndex{}
}
p.mu.Lock()
defer p.mu.Unlock()
if p.loader == nil || time.Since(p.loaded) < constants.GatewayRateLimitConfigTTL {
return p.index
}
rows, err := p.loader.ListEnabled(ctx)
p.loaded = time.Now()
if err != nil {
p.logger.Warn("加载网关限流组失败,沿用上一次配置", zap.Error(err))
return p.index
}
p.index = buildGroupIndex(rows, p.logger)
return p.index
}
func buildGroupIndex(rows []model.CarrierRateLimitGroup, logger *zap.Logger) groupIndex {
index := groupIndex{byCarrier: map[uint]map[string]GroupRule{}, global: map[string]GroupRule{}}
for _, row := range rows {
ops, err := DecodeOperations(row.Operations)
if err != nil {
logger.Warn("限流组操作列表非法,已跳过该组", zap.Uint("group_id", row.ID), zap.Error(err))
continue
}
highOps, _ := DecodeOperations(row.HighPriorityOperations)
high := map[string]bool{}
for _, op := range highOps {
high[op] = true
}
target := index.global
if row.CarrierID != nil {
if index.byCarrier[*row.CarrierID] == nil {
index.byCarrier[*row.CarrierID] = map[string]GroupRule{}
}
target = index.byCarrier[*row.CarrierID]
}
for _, op := range ops {
target[op] = GroupRule{GroupCode: row.GroupCode, MaxQPS: row.MaxQPS, ReservedQPS: row.ReservedQPS, HighPriority: high[op]}
}
}
return index
}
// DecodeOperations 解析 JSON 字符串数组形式的操作列表。
func DecodeOperations(raw []byte) ([]string, error) {
if len(raw) == 0 {
return nil, nil
}
var ops []string
if err := sonic.Unmarshal(raw, &ops); err != nil {
return nil, err
}
return ops, nil
}

View File

@@ -0,0 +1,184 @@
package gatewaylimit
import (
"context"
"strconv"
"time"
"github.com/redis/go-redis/v9"
"go.uber.org/zap"
"github.com/break/junhong_cmp_fiber/internal/gateway"
"github.com/break/junhong_cmp_fiber/pkg/constants"
)
// 分钟计数 Hash 的字段名。
const (
statFieldWouldLimit = "would_limit"
statFieldDurationSum = "duration_ms_sum"
statFieldAttempts = "attempts"
statPrefixExempt = "exempt:"
statPrefixUpstream = "upstream:"
statPrefixDuration = "duration_le:"
)
// StatFieldRateLimitWait 是等待模式下进入限流等待的次数字段(等待超时另计入 rate_limited)。
const StatFieldRateLimitWait = "rate_wait"
// durationBuckets 是耗时分桶上界(毫秒),用于近似计算 p95。
var durationBuckets = []int64{100, 300, 500, 1000, 3000, 10000, 60000}
// Guard 实现 gateway.CallGuard:每次卡网关 HTTP 尝试前按运营商账户限流,结束后写分钟计数与卡最近错误。
type Guard struct {
limiter *Limiter
resolver *CarrierResolver
classifier *Classifier
redis *redis.Client
logger *zap.Logger
}
// NewGuard 创建卡网关调用守卫。
func NewGuard(limiter *Limiter, resolver *CarrierResolver, classifier *Classifier, redisClient *redis.Client, logger *zap.Logger) *Guard {
if logger == nil {
logger = zap.NewNop()
}
return &Guard{limiter: limiter, resolver: resolver, classifier: classifier, redis: redisClient, logger: logger}
}
var _ gateway.CallGuard = (*Guard)(nil)
// BeforeAttempt 取限流名额;豁免调用直接放行。拦截模式下拿不到名额返回限流错误,本次方法调用随即结束。
func (g *Guard) BeforeAttempt(ctx context.Context, call gateway.CallInfo) error {
if g == nil || g.limiter == nil || call.ExemptSource != "" {
return nil
}
carrierID := g.carrierOf(ctx, call)
decision, err := g.limiter.Check(ctx, carrierID, call.Operation)
if err != nil {
// 限流器故障时失败开放,避免 Redis 抖动阻断全部网关调用。
g.logger.Warn("网关限流判断失败,本次放行", zap.Uint("carrier_id", carrierID), zap.String("operation", call.Operation), zap.Error(err))
return nil
}
if decision.Exceed {
g.incr(ctx, carrierID, call.Operation, statFieldWouldLimit)
}
if !decision.Allowed {
// 等待模式(NSQ 消费者):等名额直到超时;超时按限流错误返回,调用方按短延迟重排、不计失败。
if maxWait := gateway.RateLimitWait(ctx); maxWait > 0 {
g.incr(ctx, carrierID, call.Operation, StatFieldRateLimitWait)
allowed, waitErr := g.limiter.Wait(ctx, carrierID, call.Operation, maxWait)
if allowed {
return nil
}
if waitErr != nil && ctx.Err() == nil {
g.logger.Warn("网关限流等待失败,本次放行", zap.Uint("carrier_id", carrierID), zap.String("operation", call.Operation), zap.Error(waitErr))
return nil
}
}
g.incr(ctx, carrierID, call.Operation, constants.GatewayErrorRateLimited)
return gateway.NewRateLimitedError()
}
return nil
}
// AfterAttempt 写分钟计数(按结果类别与耗时分桶);查询失败时写卡该操作最近一次错误。
func (g *Guard) AfterAttempt(ctx context.Context, call gateway.CallInfo, callErr error, duration time.Duration) {
if g == nil || g.redis == nil {
return
}
ctx = context.WithoutCancel(ctx)
carrierID := g.carrierOf(ctx, call)
field := constants.GatewayResultSuccess
var classification Classification
gwErr, isGatewayErr := gateway.AsGatewayError(callErr)
if callErr != nil {
field = constants.GatewayErrorNetwork
if isGatewayErr {
field = gwErr.Category
if gwErr.Category == constants.GatewayErrorUpstream {
classification = g.classifier.Classify(ctx, call.Operation, gwErr.ProviderMessage)
field = statPrefixUpstream + classification.Category
}
}
}
key := constants.RedisGatewayCallStatKey(carrierID, call.Operation, statMinute(time.Now()))
durationMS := duration.Milliseconds()
pipe := g.redis.Pipeline()
pipe.HIncrBy(ctx, key, field, 1)
pipe.HIncrBy(ctx, key, statFieldAttempts, 1)
pipe.HIncrBy(ctx, key, statFieldDurationSum, durationMS)
pipe.HIncrBy(ctx, key, statPrefixDuration+durationBucket(durationMS), 1)
if call.ExemptSource != "" {
pipe.HIncrBy(ctx, key, statPrefixExempt+call.ExemptSource, 1)
}
pipe.Expire(ctx, key, constants.GatewayCallStatTTL)
if callErr != nil && isQueryOperation(call.Operation) && call.CardNo != "" {
errKey := constants.RedisGatewayCardLastErrorKey(call.CardNo, call.Operation)
values := map[string]any{"category": field, "at": time.Now().Format(time.RFC3339)}
if isGatewayErr {
values["provider_code"] = strconv.Itoa(gwErr.ProviderCode)
values["provider_message"] = gwErr.ProviderMessage
}
pipe.HSet(ctx, errKey, values)
pipe.Expire(ctx, errKey, constants.GatewayCardLastErrorTTL)
}
if _, err := pipe.Exec(ctx); err != nil {
g.logger.Debug("写网关调用统计失败", zap.Error(err))
}
}
// Classify 暴露分类器供停复机记录等调用方对上游错误分类。
func (g *Guard) Classify(ctx context.Context, op, message string) Classification {
if g == nil {
return Classification{Category: constants.GatewayClassBusiness}
}
return g.classifier.Classify(ctx, op, message)
}
func (g *Guard) carrierOf(ctx context.Context, call gateway.CallInfo) uint {
if call.CarrierID != 0 || g.resolver == nil {
return call.CarrierID
}
carrierID, err := g.resolver.Resolve(ctx, call.CardNo)
if err != nil {
g.logger.Debug("解析卡所属运营商账户失败,按未知账户计量", zap.String("iccid", call.CardNo), zap.Error(err))
}
return carrierID
}
func (g *Guard) incr(ctx context.Context, carrierID uint, op, field string) {
if g.redis == nil {
return
}
key := constants.RedisGatewayCallStatKey(carrierID, op, statMinute(time.Now()))
pipe := g.redis.Pipeline()
pipe.HIncrBy(ctx, key, field, 1)
pipe.Expire(ctx, key, constants.GatewayCallStatTTL)
_, _ = pipe.Exec(context.WithoutCancel(ctx))
}
func isQueryOperation(op string) bool {
return op == constants.GatewayOpQueryFlow || op == constants.GatewayOpQueryStatus || op == constants.GatewayOpQueryRealname
}
// statMinute 返回上海时区的分钟标识(yyyyMMddHHmm)。
func statMinute(t time.Time) string {
return t.In(shanghai()).Format("200601021504")
}
func durationBucket(ms int64) string {
for _, upper := range durationBuckets {
if ms <= upper {
return strconv.FormatInt(upper, 10)
}
}
return "inf"
}
func shanghai() *time.Location {
location, err := time.LoadLocation("Asia/Shanghai")
if err != nil {
return time.FixedZone("CST", 8*3600)
}
return location
}

View File

@@ -0,0 +1,238 @@
package gatewaylimit
import (
"context"
"crypto/rand"
"encoding/hex"
"strconv"
"time"
"github.com/redis/go-redis/v9"
"github.com/break/junhong_cmp_fiber/pkg/constants"
)
// slidingWindowScript 在一次原子执行内判断账户组窗口与可选的全局组窗口。
//
// KEYS[1] 账户组窗口 ZSET;KEYS[2] 账户组高优先级等待计数;KEYS[3] 全局组窗口 ZSET(无全局组时传空串占位)。
// ARGV:now_ms、window_ms、max_qps、reserved_qps、is_high(0/1)、global_max(0 表示无全局组)、member、dry_run(0/1)。
// 返回 {allowed(1/0), wait_ms}:dry_run=1(只统计)时总是记录本次请求,allowed=0 表示"本应被限流"。
const slidingWindowLua = `
local now = tonumber(ARGV[1])
local window = tonumber(ARGV[2])
local maxQPS = tonumber(ARGV[3])
local reserved = tonumber(ARGV[4])
local isHigh = tonumber(ARGV[5])
local globalMax = tonumber(ARGV[6])
local member = ARGV[7]
local dryRun = tonumber(ARGV[8])
local useGlobal = globalMax > 0 and KEYS[3] ~= ''
redis.call('ZREMRANGEBYSCORE', KEYS[1], '-inf', now - window)
if useGlobal then
redis.call('ZREMRANGEBYSCORE', KEYS[3], '-inf', now - window)
end
local limit = maxQPS
if isHigh == 0 then
limit = maxQPS - reserved
if reserved > 0 then
local waiting = tonumber(redis.call('GET', KEYS[2]) or '0')
if waiting > 0 then
limit = 0
end
end
end
local allowed = 1
local wait = 0
local count = redis.call('ZCARD', KEYS[1])
if count >= limit then
allowed = 0
local oldest = redis.call('ZRANGE', KEYS[1], 0, 0, 'WITHSCORES')
if oldest[2] then
wait = tonumber(oldest[2]) + window - now
else
wait = window
end
end
if allowed == 1 and useGlobal then
local gcount = redis.call('ZCARD', KEYS[3])
if gcount >= globalMax then
allowed = 0
local goldest = redis.call('ZRANGE', KEYS[3], 0, 0, 'WITHSCORES')
if goldest[2] then
wait = tonumber(goldest[2]) + window - now
else
wait = window
end
end
end
if allowed == 1 or dryRun == 1 then
redis.call('ZADD', KEYS[1], now, member)
redis.call('PEXPIRE', KEYS[1], window * 2)
if useGlobal then
redis.call('ZADD', KEYS[3], now, member)
redis.call('PEXPIRE', KEYS[3], window * 2)
end
end
if wait < 1 and allowed == 0 then
wait = 1
end
return {allowed, wait}
`
var slidingWindowScript = redis.NewScript(slidingWindowLua)
// Decision 是一次限流判断的结果。
type Decision struct {
Allowed bool // 是否放行(只统计模式下恒为 true)
Exceed bool // 是否超过上限(只统计模式下表示"本应被限流")
Wait time.Duration // 拦截时建议的等待时长
}
// Limiter 是基于 Redis 滑动窗口的运营商账户限流器,跨全部进程共享。
type Limiter struct {
redis *redis.Client
groups *GroupProvider
settings *SettingsProvider
}
// NewLimiter 创建限流器。
func NewLimiter(redisClient *redis.Client, groups *GroupProvider, settings *SettingsProvider) *Limiter {
return &Limiter{redis: redisClient, groups: groups, settings: settings}
}
// Check 立即判断一次请求能否放行,不等待。拦截模式下超过上限返回 Allowed=false;
// 只统计模式下总是放行,Exceed 表示本应被限流。Redis 故障时放行(失败开放),避免限流故障阻断业务。
func (l *Limiter) Check(ctx context.Context, carrierID uint, op string) (Decision, error) {
settings := l.settings.Current(ctx)
account, accountOK, global, globalOK := l.groups.Resolve(ctx, carrierID, op)
if !accountOK {
account = GroupRule{GroupCode: defaultGroupCode + ":" + op, MaxQPS: settings.DefaultQPS}
}
globalKey, globalMax := "", 0
if globalOK {
globalKey, globalMax = constants.RedisGatewayGlobalQPSKey(global.GroupCode), global.MaxQPS
}
isHigh, dryRun := 0, 1
if account.HighPriority {
isHigh = 1
}
if settings.Enforce() {
dryRun = 0
}
now := time.Now().UnixMilli()
result, err := slidingWindowScript.Run(ctx, l.redis,
[]string{constants.RedisGatewayQPSKey(carrierID, account.GroupCode), constants.RedisGatewayQPSHighWaitingKey(carrierID, account.GroupCode), globalKey},
now, constants.GatewayRateLimitWindow.Milliseconds(), account.MaxQPS, account.ReservedQPS, isHigh, globalMax, windowMember(now), dryRun,
).Int64Slice()
if err != nil || len(result) != 2 {
return Decision{Allowed: true}, err
}
exceed := result[0] == 0
decision := Decision{Allowed: !exceed || dryRun == 1, Exceed: exceed}
if exceed && dryRun == 0 {
decision.Wait = time.Duration(result[1]) * time.Millisecond
}
return decision, nil
}
// Wait 在拦截模式下按建议时长等待名额,总等待不超过 maxWait;超时返回 false。
// 高优先级操作等待期间登记等待计数,使同组普通操作拿不到新名额。本 Change 中 Asynq 任务不使用该方法。
func (l *Limiter) Wait(ctx context.Context, carrierID uint, op string, maxWait time.Duration) (bool, error) {
deadline := time.Now().Add(maxWait)
registered := false
var waitingKey string
defer func() {
if registered {
l.redis.Decr(context.WithoutCancel(ctx), waitingKey)
}
}()
for {
decision, err := l.Check(ctx, carrierID, op)
if err != nil || decision.Allowed {
return true, err
}
if !registered {
account, accountOK, _, _ := l.groups.Resolve(ctx, carrierID, op)
if accountOK && account.HighPriority && account.ReservedQPS > 0 {
waitingKey = constants.RedisGatewayQPSHighWaitingKey(carrierID, account.GroupCode)
pipe := l.redis.TxPipeline()
pipe.Incr(ctx, waitingKey)
pipe.Expire(ctx, waitingKey, 10*time.Second)
if _, err := pipe.Exec(ctx); err == nil {
registered = true
}
}
}
sleep := decision.Wait
if remaining := time.Until(deadline); remaining <= 0 {
return false, nil
} else if sleep > remaining {
sleep = remaining
}
timer := time.NewTimer(sleep)
select {
case <-ctx.Done():
timer.Stop()
return false, ctx.Err()
case <-timer.C:
}
}
}
// windowMember 生成窗口内唯一成员,避免同毫秒请求互相覆盖。
func windowMember(nowMS int64) string {
buf := make([]byte, 6)
_, _ = rand.Read(buf)
return strconv.FormatInt(nowMS, 10) + "-" + hex.EncodeToString(buf)
}
// Budget 返回运营商账户各查询类操作所属限流组本秒剩余名额,供分发器按预计调用数领取。
// groups 为限流组 → 本秒剩余;opGroup 为操作 → 限流组。只统计模式下返回 limited=false(不按 QPS 限制领取)。
// 全局组只在账户组之外再收紧:剩余名额取两者较小值。
func (l *Limiter) Budget(ctx context.Context, carrierID uint, ops []string) (groups map[string]int, opGroup map[string]string, limited bool) {
if l == nil || l.redis == nil {
return nil, nil, false
}
settings := l.settings.Current(ctx)
if !settings.Enforce() {
return nil, nil, false
}
groups, opGroup = map[string]int{}, map[string]string{}
now := time.Now().UnixMilli()
since := now - constants.GatewayRateLimitWindow.Milliseconds()
for _, op := range ops {
account, accountOK, global, globalOK := l.groups.Resolve(ctx, carrierID, op)
if !accountOK {
account = GroupRule{GroupCode: defaultGroupCode + ":" + op, MaxQPS: settings.DefaultQPS}
}
code := account.GroupCode
opGroup[op] = code
if _, seen := groups[code]; seen {
continue
}
limit := account.MaxQPS
if !account.HighPriority {
limit -= account.ReservedQPS
}
used, err := l.redis.ZCount(ctx, constants.RedisGatewayQPSKey(carrierID, code), strconv.FormatInt(since, 10), "+inf").Result()
if err != nil {
return nil, nil, false
}
remaining := limit - int(used)
if globalOK {
globalUsed, gerr := l.redis.ZCount(ctx, constants.RedisGatewayGlobalQPSKey(global.GroupCode), strconv.FormatInt(since, 10), "+inf").Result()
if gerr == nil && global.MaxQPS-int(globalUsed) < remaining {
remaining = global.MaxQPS - int(globalUsed)
}
}
if remaining < 0 {
remaining = 0
}
groups[code] = remaining
}
return groups, opGroup, true
}

View File

@@ -0,0 +1,61 @@
package gatewaylimit
import (
"context"
stderrors "errors"
"time"
"github.com/hibiken/asynq"
"go.uber.org/zap"
"github.com/break/junhong_cmp_fiber/internal/store/postgres"
)
// partitionDropLookbackMonths 是每次清理向前检查的过期月数,覆盖维护任务停摆一段时间后的补删。
const partitionDropLookbackMonths = 12
// PartitionMaintainer 维护停复机记录按月分区:提前创建未来分区,按保留期整区删除过期分区。
type PartitionMaintainer struct {
store *postgres.CardStopResumeRecordStore
settings *SettingsProvider
logger *zap.Logger
}
// NewPartitionMaintainer 创建分区维护任务。
func NewPartitionMaintainer(store *postgres.CardStopResumeRecordStore, settings *SettingsProvider, logger *zap.Logger) *PartitionMaintainer {
if logger == nil {
logger = zap.NewNop()
}
return &PartitionMaintainer{store: store, settings: settings, logger: logger}
}
// Handle 是 Asynq 定时任务入口;任一分区操作失败时写错误告警并返回错误,由 Asynq 按既有规则重试。
func (m *PartitionMaintainer) Handle(ctx context.Context, _ *asynq.Task) error {
if m == nil || m.store == nil {
return stderrors.New("停复机记录分区维护未配置")
}
settings := m.settings.Current(ctx)
monthStart := currentMonthStart(time.Now())
var failed error
for i := 0; i <= settings.RecordPremakeMonth; i++ {
month := monthStart.AddDate(0, i, 0)
if err := m.store.EnsureMonthPartition(ctx, month); err != nil {
m.logger.Error("创建停复机记录分区失败,记录将落入默认分区", zap.String("month", month.Format("200601")), zap.Error(err))
failed = err
}
}
for k := 0; k < partitionDropLookbackMonths; k++ {
month := monthStart.AddDate(0, -(settings.RecordRetention + k), 0)
if err := m.store.DropMonthPartition(ctx, month); err != nil {
m.logger.Error("删除过期停复机记录分区失败", zap.String("month", month.Format("200601")), zap.Error(err))
failed = err
}
}
return failed
}
// currentMonthStart 返回上海时区当月 1 日 0 点。
func currentMonthStart(now time.Time) time.Time {
local := now.In(shanghai())
return time.Date(local.Year(), local.Month(), 1, 0, 0, 0, 0, shanghai())
}

View File

@@ -0,0 +1,53 @@
package gatewaylimit
import (
"context"
"strconv"
"github.com/redis/go-redis/v9"
"gorm.io/gorm"
"github.com/break/junhong_cmp_fiber/pkg/constants"
)
// CarrierResolver 按 ICCID 解析卡所属的本系统运营商账户(tb_iot_card.carrier_id),结果缓存在 Redis。
type CarrierResolver struct {
db *gorm.DB
redis *redis.Client
}
// NewCarrierResolver 创建运营商账户解析器。
func NewCarrierResolver(db *gorm.DB, redisClient *redis.Client) *CarrierResolver {
return &CarrierResolver{db: db, redis: redisClient}
}
// Resolve 返回 ICCID 对应卡的运营商账户 ID;卡不存在时返回 0、nil,由调用方按未知账户处理。
func (r *CarrierResolver) Resolve(ctx context.Context, iccid string) (uint, error) {
if r == nil || iccid == "" {
return 0, nil
}
cacheKey := constants.RedisGatewayCardCarrierKey(iccid)
if r.redis != nil {
if raw, err := r.redis.Get(ctx, cacheKey).Result(); err == nil {
if value, parseErr := strconv.ParseUint(raw, 10, 64); parseErr == nil {
return uint(value), nil
}
}
}
if r.db == nil {
return 0, nil
}
var carrierIDs []uint
err := r.db.WithContext(ctx).Table("tb_iot_card").
Where("iccid = ? AND deleted_at IS NULL", iccid).Limit(1).Pluck("carrier_id", &carrierIDs).Error
if err != nil {
return 0, err
}
if len(carrierIDs) == 0 {
return 0, nil
}
if r.redis != nil {
_ = r.redis.Set(ctx, cacheKey, strconv.FormatUint(uint64(carrierIDs[0]), 10), constants.GatewayCardCarrierCacheTTL).Err()
}
return carrierIDs[0], nil
}

View File

@@ -0,0 +1,83 @@
package gatewaylimit
import (
"context"
"crypto/sha1"
"encoding/hex"
"time"
"github.com/hibiken/asynq"
"github.com/redis/go-redis/v9"
"go.uber.org/zap"
"gorm.io/gorm"
"github.com/break/junhong_cmp_fiber/internal/infrastructure/systemconfig"
"github.com/break/junhong_cmp_fiber/internal/store/postgres"
"github.com/break/junhong_cmp_fiber/pkg/constants"
)
// Runtime 汇总一个进程内的网关限流组件,API 与 Worker 各自创建一份,限流窗口经 Redis 跨进程共享。
type Runtime struct {
Settings *SettingsProvider
Groups *GroupProvider
Classifier *Classifier
Limiter *Limiter
Guard *Guard
Recorder *StopResumeRecorder
// TerminalPolling 停复机命中"自动关闭轮询"终态规则时关闭该卡轮询。
TerminalPolling *TerminalPollingDisabler
redisClient *redis.Client
logger *zap.Logger
}
// NewRuntime 创建网关限流运行组件;配置读取使用独立注册表,与后台系统配置共享同一份数据库与缓存数据。
func NewRuntime(db *gorm.DB, redisClient *redis.Client, logger *zap.Logger) *Runtime {
if logger == nil {
logger = zap.NewNop()
}
registry := systemconfig.NewRegistry()
RegisterConfigDefinitions(registry, logger)
var cache systemconfig.Cache
if redisClient != nil {
cache = systemconfig.NewRedisCache(redisClient)
}
reader := systemconfig.NewReader(db, registry, cache, systemconfig.NewLogAlertSink(logger))
settings := NewSettingsProvider(reader, logger)
groups := NewGroupProvider(postgres.NewCarrierRateLimitGroupStore(db), logger)
classifier := NewClassifier(postgres.NewGatewayErrorClassificationStore(db), logger)
limiter := NewLimiter(redisClient, groups, settings)
guard := NewGuard(limiter, NewCarrierResolver(db, redisClient), classifier, redisClient, logger)
recorder := NewStopResumeRecorder(postgres.NewCardStopResumeRecordStore(db), classifier, logger)
terminal := NewTerminalPollingDisabler(db, classifier, logger)
return &Runtime{Settings: settings, Groups: groups, Classifier: classifier, Limiter: limiter, Guard: guard, Recorder: recorder,
TerminalPolling: terminal, redisClient: redisClient, logger: logger}
}
// TrackDeferral 记录 Asynq 任务因限流被推迟;同一任务首次被限流起累计推迟超过阈值时写一次错误告警,任务继续重试。
// 任务以"类型 + 载荷摘要"标识,记录 1 天后自动过期。
func (r *Runtime) TrackDeferral(task *asynq.Task) {
if r == nil || r.redisClient == nil || task == nil {
return
}
ctx := context.Background()
sum := sha1.Sum(task.Payload())
taskKey := task.Type() + ":" + hex.EncodeToString(sum[:])
key := constants.RedisRateLimitDeferStartKey(taskKey)
now := time.Now()
if ok, err := r.redisClient.SetNX(ctx, key, now.Unix(), 24*time.Hour).Result(); err != nil || ok {
return
}
startedUnix, err := r.redisClient.Get(ctx, key).Int64()
if err != nil {
return
}
deferred := now.Sub(time.Unix(startedUnix, 0))
if deferred < r.Settings.Current(ctx).DeferAlert {
return
}
if alerted, _ := r.redisClient.SetNX(ctx, key+":alerted", 1, 24*time.Hour).Result(); alerted {
r.logger.Error("异步任务因运营商账户网关限流累计推迟超过告警阈值,任务继续重试",
zap.String("task_type", task.Type()), zap.String("task_key", taskKey), zap.Duration("deferred", deferred))
}
}

View File

@@ -0,0 +1,136 @@
// Package gatewaylimit 实现按运营商账户(本系统 tb_carrier)的卡网关调用限流、错误分类与调用统计。
//
// 账户口径:限流单位是本系统的运营商账户(卡上的 carrier_id),与网关客户端自身的 appId 无关。
// 调用方已知卡所属账户时用 gateway.WithCarrier 传入;否则守卫按 ICCID 查本系统卡表解析。
package gatewaylimit
import (
"context"
stderrors "errors"
"strconv"
"sync"
"time"
"go.uber.org/zap"
"github.com/break/junhong_cmp_fiber/internal/infrastructure/systemconfig"
"github.com/break/junhong_cmp_fiber/pkg/constants"
)
// RegisterConfigDefinitions 向受控系统配置注册表注册网关限流相关配置;已注册的 Key 跳过。
func RegisterConfigDefinitions(registry *systemconfig.Registry, logger *zap.Logger) {
if registry == nil {
return
}
minOne, minZero := int64(1), int64(0)
maxQPS, maxDelay, maxAlert, maxMonths := int64(10000), int64(3600), int64(86400*7), int64(120)
definitions := []systemconfig.Definition{
{Key: constants.SystemConfigPollingRateLimitMode, Module: constants.SystemConfigModulePolling, ValueType: constants.SystemConfigTypeString, DefaultValue: constants.RateLimitModeObserve, Description: "网关限流模式:observe 只统计不拦截,enforce 拦截", EnumValues: []string{constants.RateLimitModeObserve, constants.RateLimitModeEnforce}},
{Key: constants.SystemConfigPollingGatewayDefaultQPS, Module: constants.SystemConfigModulePolling, ValueType: constants.SystemConfigTypeInt, DefaultValue: "10", Description: "未被启用限流组覆盖的卡网关操作的默认每秒请求上限", Min: &minOne, Max: &maxQPS},
{Key: constants.SystemConfigPollingRateLimitedRetryDelay, Module: constants.SystemConfigModulePolling, ValueType: constants.SystemConfigTypeInt, DefaultValue: "5", Description: "Asynq 任务遇限流后的固定重试延迟(秒)", Min: &minOne, Max: &maxDelay},
{Key: constants.SystemConfigPollingRateLimitDeferAlert, Module: constants.SystemConfigModulePolling, ValueType: constants.SystemConfigTypeInt, DefaultValue: "600", Description: "同一任务因限流累计推迟超过该时长(秒)时写告警", Min: &minOne, Max: &maxAlert},
{Key: constants.SystemConfigPollingStopResumeRecordRetention, Module: constants.SystemConfigModulePolling, ValueType: constants.SystemConfigTypeInt, DefaultValue: "6", Description: "停复机记录保留月数,到期整区删除", Min: &minOne, Max: &maxMonths},
{Key: constants.SystemConfigPollingStopResumeRecordPremake, Module: constants.SystemConfigModulePolling, ValueType: constants.SystemConfigTypeInt, DefaultValue: "3", Description: "停复机记录提前创建未来分区的月数", Min: &minZero, Max: &maxMonths},
}
for _, definition := range definitions {
if _, exists := registry.Get(definition.Key); exists {
continue
}
if err := registry.Register(definition); err != nil && logger != nil {
logger.Error("注册网关限流系统配置失败,将使用代码默认值", zap.String("config_key", definition.Key), zap.Error(err))
}
}
}
// ConfigReader 读取已注册的受控系统配置。
type ConfigReader interface {
Get(ctx context.Context, key string) (string, error)
}
// Settings 是网关限流运行参数快照。
type Settings struct {
Mode string
DefaultQPS int
RetryDelay time.Duration
DeferAlert time.Duration
RecordRetention int
RecordPremakeMonth int
}
// Enforce 判断当前是否为拦截模式。
func (s Settings) Enforce() bool {
return s.Mode == constants.RateLimitModeEnforce
}
// defaultSettings 是配置读取失败时的安全默认值:只统计、不拦截。
func defaultSettings() Settings {
return Settings{Mode: constants.RateLimitModeObserve, DefaultQPS: 10, RetryDelay: 5 * time.Second, DeferAlert: 10 * time.Minute, RecordRetention: 6, RecordPremakeMonth: 3}
}
// SettingsProvider 在进程内缓存限流参数,超过 TTL 重新读取,后台修改数秒内生效。
type SettingsProvider struct {
reader ConfigReader
logger *zap.Logger
mu sync.Mutex
cached Settings
loaded time.Time
}
// NewSettingsProvider 创建限流参数提供者;reader 为 nil 时始终返回安全默认值。
func NewSettingsProvider(reader ConfigReader, logger *zap.Logger) *SettingsProvider {
if logger == nil {
logger = zap.NewNop()
}
return &SettingsProvider{reader: reader, logger: logger, cached: defaultSettings()}
}
// Current 返回当前限流参数;读取失败时保留上一次成功值。
func (p *SettingsProvider) Current(ctx context.Context) Settings {
if p == nil {
return defaultSettings()
}
p.mu.Lock()
defer p.mu.Unlock()
if p.reader == nil || time.Since(p.loaded) < constants.GatewayRateLimitConfigTTL {
return p.cached
}
next, err := p.load(ctx)
p.loaded = time.Now()
if err != nil {
p.logger.Warn("读取网关限流配置失败,沿用上一次配置", zap.Error(err))
return p.cached
}
p.cached = next
return next
}
func (p *SettingsProvider) load(ctx context.Context) (Settings, error) {
settings := defaultSettings()
mode, err := p.reader.Get(ctx, constants.SystemConfigPollingRateLimitMode)
if err != nil {
return settings, err
}
settings.Mode = mode
ints := []struct {
key string
set func(int)
}{
{constants.SystemConfigPollingGatewayDefaultQPS, func(v int) { settings.DefaultQPS = v }},
{constants.SystemConfigPollingRateLimitedRetryDelay, func(v int) { settings.RetryDelay = time.Duration(v) * time.Second }},
{constants.SystemConfigPollingRateLimitDeferAlert, func(v int) { settings.DeferAlert = time.Duration(v) * time.Second }},
{constants.SystemConfigPollingStopResumeRecordRetention, func(v int) { settings.RecordRetention = v }},
{constants.SystemConfigPollingStopResumeRecordPremake, func(v int) { settings.RecordPremakeMonth = v }},
}
for _, item := range ints {
raw, err := p.reader.Get(ctx, item.key)
if err != nil {
return settings, err
}
value, err := strconv.Atoi(raw)
if err != nil {
return settings, stderrors.New("网关限流整数配置非法")
}
item.set(value)
}
return settings, nil
}

View File

@@ -0,0 +1,86 @@
package gatewaylimit
import (
"context"
"strconv"
"time"
"go.uber.org/zap"
"github.com/break/junhong_cmp_fiber/internal/gateway"
"github.com/break/junhong_cmp_fiber/internal/model"
"github.com/break/junhong_cmp_fiber/internal/store/postgres"
"github.com/break/junhong_cmp_fiber/pkg/constants"
)
// StopResumeRecordInput 描述一次实际调用上游停机或复机的结果。
type StopResumeRecordInput struct {
CardID uint
ICCID string
CarrierID uint
Action string // constants.StopResumeActionStop / StopResumeActionStart
Source string // 来源:豁免来源或审计来源
Scene string
Reason string // 停机原因
CommandID string // 同一次停复机命令的多次尝试共用
Attempt int
CallErr error
Duration time.Duration
}
// StopResumeRecorder 把每次停复机上游调用写入停复机记录表;写入失败只记日志与计数,不影响业务回写。
type StopResumeRecorder struct {
store *postgres.CardStopResumeRecordStore
classifier *Classifier
logger *zap.Logger
}
// NewStopResumeRecorder 创建停复机记录写入器。
func NewStopResumeRecorder(store *postgres.CardStopResumeRecordStore, classifier *Classifier, logger *zap.Logger) *StopResumeRecorder {
if logger == nil {
logger = zap.NewNop()
}
return &StopResumeRecorder{store: store, classifier: classifier, logger: logger}
}
// Record 写入一条停复机记录;限流拒绝(请求未发出)不写。
func (r *StopResumeRecorder) Record(ctx context.Context, input StopResumeRecordInput) {
if r == nil || r.store == nil || gateway.IsRateLimited(input.CallErr) {
return
}
row := &model.CardStopResumeRecord{
CardID: input.CardID, ICCID: input.ICCID, CarrierID: input.CarrierID, Action: input.Action,
Source: input.Source, Scene: input.Scene, Reason: input.Reason, CommandID: input.CommandID,
Attempt: input.Attempt, Result: constants.StopResumeRecordSuccess,
DurationMS: int(input.Duration.Milliseconds()), CreatedAt: time.Now(),
}
if input.CallErr != nil {
row.Result = constants.StopResumeRecordFailed
if gwErr, ok := gateway.AsGatewayError(input.CallErr); ok {
row.ErrorCategory = gwErr.Category
switch gwErr.Category {
case constants.GatewayErrorTimeout, constants.GatewayErrorNetwork:
// 超时或连接失败时上游可能已受理,按结果未知记录。
row.Result = constants.StopResumeRecordUnknown
case constants.GatewayErrorUpstream:
op := constants.GatewayOpStop
if input.Action == constants.StopResumeActionStart {
op = constants.GatewayOpStart
}
classification := r.classifier.Classify(ctx, op, gwErr.ProviderMessage)
row.ErrorCategory = classification.Category
if classification.Category == constants.GatewayClassAlreadyTarget {
row.Result = constants.StopResumeRecordAlreadyTarget
}
row.ProviderCode = strconv.Itoa(gwErr.ProviderCode)
row.ProviderMessage = gwErr.ProviderMessage
}
}
}
writeCtx, cancel := context.WithTimeout(context.WithoutCancel(ctx), 3*time.Second)
defer cancel()
if err := r.store.Create(writeCtx, row); err != nil {
r.logger.Warn("写停复机记录失败,不影响停复机结果回写",
zap.Uint("card_id", input.CardID), zap.String("action", input.Action), zap.String("metric", "stop_resume_record.write_failed"), zap.Error(err))
}
}

View File

@@ -0,0 +1,113 @@
package gatewaylimit
import (
"context"
"strconv"
"strings"
"time"
"go.uber.org/zap"
"gorm.io/gorm"
"github.com/break/junhong_cmp_fiber/internal/gateway"
auditinfra "github.com/break/junhong_cmp_fiber/internal/infrastructure/audit"
"github.com/break/junhong_cmp_fiber/internal/model"
"github.com/break/junhong_cmp_fiber/pkg/constants"
)
// TerminalPollingDisabler 在停复机调用命中"自动关闭轮询"的终态规则时关闭该卡自身轮询:
// 不级联设备与设备下其他卡;同事务回写网关扩展状态并写审计。
type TerminalPollingDisabler struct {
db *gorm.DB
classifier *Classifier
audit *auditinfra.Writer
logger *zap.Logger
}
// NewTerminalPollingDisabler 创建终态关闭轮询处理器;db 或 classifier 为 nil 时不处理。
func NewTerminalPollingDisabler(db *gorm.DB, classifier *Classifier, logger *zap.Logger) *TerminalPollingDisabler {
if logger == nil {
logger = zap.NewNop()
}
return &TerminalPollingDisabler{db: db, classifier: classifier, logger: logger}
}
// SetAuditWriter 注入统一审计;未注入时只关闭轮询并记日志。
func (d *TerminalPollingDisabler) SetAuditWriter(writer *auditinfra.Writer) {
if d != nil {
d.audit = writer
}
}
// HandleCommandError 判断停复机上游失败是否命中"自动关闭轮询"规则;命中时关闭该卡轮询并返回 true,
// 调用方据此不再重试。action 为 constants.StopResumeActionStop / StopResumeActionStart。
func (d *TerminalPollingDisabler) HandleCommandError(ctx context.Context, card *model.IotCard, action string, callErr error) bool {
if d == nil || d.db == nil || d.classifier == nil || card == nil || card.ID == 0 || callErr == nil {
return false
}
gwErr, ok := gateway.AsGatewayError(callErr)
if !ok || gwErr.Category != constants.GatewayErrorUpstream {
return false
}
op := constants.GatewayOpStop
if action == constants.StopResumeActionStart {
op = constants.GatewayOpStart
}
classification := d.classifier.Classify(ctx, op, gwErr.ProviderMessage)
if !classification.AutoDisablePolling {
return false
}
extend := terminalGatewayExtend(gwErr.ProviderMessage, card.GatewayExtend)
if err := d.disable(ctx, card, extend, classification, gwErr.ProviderMessage); err != nil {
d.logger.Error("上游终态自动关闭卡轮询失败", zap.Uint("card_id", card.ID), zap.String("iccid", card.ICCID),
zap.String("metric", "stop_resume.terminal_disable_polling.failed"), zap.Error(err))
return true
}
d.logger.Warn("停复机命中终态规则,已关闭该卡轮询",
zap.Uint("card_id", card.ID), zap.String("iccid", card.ICCID), zap.String("action", action),
zap.Uint("rule_id", classification.RuleID), zap.String("keyword", classification.Keyword),
zap.String("gateway_extend", extend), zap.String("metric", "stop_resume.terminal_disable_polling"))
return true
}
func (d *TerminalPollingDisabler) disable(ctx context.Context, card *model.IotCard, extend string, classification Classification, message string) error {
writeCtx, cancel := context.WithTimeout(context.WithoutCancel(ctx), 5*time.Second)
defer cancel()
return d.db.WithContext(writeCtx).Transaction(func(tx *gorm.DB) error {
updates := map[string]any{"enable_polling": false, "gateway_extend": extend, "updated_at": time.Now()}
if err := tx.Model(&model.IotCard{}).Where("id = ?", card.ID).Updates(updates).Error; err != nil {
return err
}
if d.audit == nil {
return nil
}
id := strconv.FormatUint(uint64(card.ID), 10)
return d.audit.Append(ctx, tx, auditinfra.AppendInput{
ActionCode: constants.AuditActionIotCardTerminalPollingDisabled,
Summary: "上游返回终态,自动关闭 IoT 卡轮询",
Result: constants.AuditResultSuccess,
Metadata: map[string]any{"rule_id": classification.RuleID, "keyword": classification.Keyword,
"provider_message": message},
Resources: []auditinfra.ResourceInput{{
Type: constants.AuditResourceIotCard, ID: &id, Key: auditinfra.IotCardResourceKey(card), DisplayName: card.ICCID,
Relation: constants.AuditResourceRelationPrimary, Role: constants.AuditResourceRoleIotCardTarget,
IdentitySnapshot: auditinfra.IotCardIdentitySnapshot(card),
BeforeData: map[string]any{"enable_polling": card.EnablePolling, "gateway_extend": card.GatewayExtend},
AfterData: map[string]any{"enable_polling": false, "gateway_extend": extend},
SubjectVisibility: constants.AuditSubjectResult, SubjectSummary: "IoT 卡已因上游终态关闭轮询",
}},
})
})
}
// terminalGatewayExtend 按上游返回推断网关扩展状态:已销户、风险停机;无法推断时保留原值。
func terminalGatewayExtend(message, current string) string {
switch {
case strings.Contains(message, "销户"):
return constants.GatewayCardExtendCancelled
case strings.Contains(message, "风险"):
return constants.GatewayCardExtendRiskStop
default:
return current
}
}

View File

@@ -0,0 +1,215 @@
// Package nsqbus 封装卡轮询使用的 NSQ 生产者与按 topic 动态订阅的消费者管理器。
// NSQ 只承担执行分发与消费者并发控制,业务状态与重试全部在 Redis(add-carrier-polling-scheduler)。
package nsqbus
import (
"context"
"strings"
"sync"
"time"
"github.com/nsqio/go-nsq"
"go.uber.org/zap"
)
// Producer 是 nsqd 生产者。
type Producer struct {
producer *nsq.Producer
}
// NewProducer 连接 nsqd TCP 地址创建生产者。
func NewProducer(address string, logger *zap.Logger) (*Producer, error) {
cfg := nsq.NewConfig()
producer, err := nsq.NewProducer(address, cfg)
if err != nil {
return nil, err
}
producer.SetLogger(nsqLogger{logger: logger}, nsq.LogLevelWarning)
return &Producer{producer: producer}, nil
}
// Publish 同步发布一条消息;topic 不存在时由 nsqd 自动创建。
func (p *Producer) Publish(topic string, body []byte) error {
return p.producer.Publish(topic, body)
}
// Stop 停止生产者。
func (p *Producer) Stop() {
if p != nil && p.producer != nil {
p.producer.Stop()
}
}
// Handler 处理一条消息;返回后由管理器 FIN。消息处理期间可调用 touch 延长 NSQ 消息超时。
type Handler func(ctx context.Context, body []byte, touch func())
// ConsumerConfig 是消费者参数。
type ConsumerConfig struct {
NSQDAddress string
NSQLookupdAddresses []string
MsgTimeout time.Duration
MaxAttempts uint16
}
// ConsumerManager 按 topic 动态订阅与取消订阅;每个 topic 一个消费者,max_in_flight 控制本进程对该 topic 的并发。
type ConsumerManager struct {
cfg ConsumerConfig
channel string
handler Handler
logger *zap.Logger
mu sync.Mutex
consumers map[string]*subscription
ctx context.Context
cancel context.CancelFunc
inflight sync.WaitGroup
}
type subscription struct {
consumer *nsq.Consumer
maxInFlight int
}
// NewConsumerManager 创建消费者管理器。
func NewConsumerManager(cfg ConsumerConfig, channel string, handler Handler, logger *zap.Logger) *ConsumerManager {
if logger == nil {
logger = zap.NewNop()
}
ctx, cancel := context.WithCancel(context.Background())
return &ConsumerManager{cfg: cfg, channel: channel, handler: handler, logger: logger,
consumers: map[string]*subscription{}, ctx: ctx, cancel: cancel}
}
// Sync 使订阅集合与 desired(topic → max_in_flight)一致:新增订阅、取消多余订阅、调整 max_in_flight。
func (m *ConsumerManager) Sync(desired map[string]int) {
m.mu.Lock()
defer m.mu.Unlock()
if m.ctx.Err() != nil {
return
}
for topic, sub := range m.consumers {
if _, keep := desired[topic]; !keep {
sub.consumer.Stop()
delete(m.consumers, topic)
m.logger.Info("取消订阅卡任务 topic", zap.String("topic", topic))
}
}
for topic, maxInFlight := range desired {
if maxInFlight <= 0 {
maxInFlight = 1
}
if sub, ok := m.consumers[topic]; ok {
if sub.maxInFlight != maxInFlight {
sub.consumer.ChangeMaxInFlight(maxInFlight)
sub.maxInFlight = maxInFlight
}
continue
}
consumer, err := m.subscribe(topic, maxInFlight)
if err != nil {
m.logger.Warn("订阅卡任务 topic 失败,下一轮重试", zap.String("topic", topic), zap.Error(err))
continue
}
m.consumers[topic] = &subscription{consumer: consumer, maxInFlight: maxInFlight}
m.logger.Info("订阅卡任务 topic", zap.String("topic", topic), zap.Int("max_in_flight", maxInFlight))
}
}
func (m *ConsumerManager) subscribe(topic string, maxInFlight int) (*nsq.Consumer, error) {
cfg := nsq.NewConfig()
cfg.MaxInFlight = maxInFlight
if m.cfg.MsgTimeout > 0 {
cfg.MsgTimeout = m.cfg.MsgTimeout
}
if m.cfg.MaxAttempts > 0 {
cfg.MaxAttempts = m.cfg.MaxAttempts
}
consumer, err := nsq.NewConsumer(topic, m.channel, cfg)
if err != nil {
return nil, err
}
consumer.SetLogger(nsqLogger{logger: m.logger}, nsq.LogLevelWarning)
handlerCtx := m.ctx // subscribe 在持锁时调用,捕获当前处理上下文
consumer.AddConcurrentHandlers(nsq.HandlerFunc(func(message *nsq.Message) error {
message.DisableAutoResponse()
m.inflight.Add(1)
defer m.inflight.Done()
defer message.Finish()
m.handler(handlerCtx, message.Body, message.Touch)
return nil
}), maxInFlight)
if len(m.cfg.NSQLookupdAddresses) > 0 {
err = consumer.ConnectToNSQLookupds(m.cfg.NSQLookupdAddresses)
} else {
err = consumer.ConnectToNSQD(m.cfg.NSQDAddress)
}
if err != nil {
consumer.Stop()
return nil, err
}
return consumer, nil
}
// Topics 返回当前已订阅的 topic。
func (m *ConsumerManager) Topics() []string {
m.mu.Lock()
defer m.mu.Unlock()
topics := make([]string, 0, len(m.consumers))
for topic := range m.consumers {
topics = append(topics, topic)
}
return topics
}
// StopAll 停止全部订阅;在宽限期内等待执行中的消息完成并 FIN,超过宽限期后取消处理上下文。
func (m *ConsumerManager) StopAll(grace time.Duration) {
m.mu.Lock()
for topic, sub := range m.consumers {
sub.consumer.Stop()
delete(m.consumers, topic)
}
m.mu.Unlock()
done := make(chan struct{})
go func() {
m.inflight.Wait()
close(done)
}()
select {
case <-done:
case <-time.After(grace):
m.logger.Warn("优雅退出宽限期内仍有卡任务未完成,取消执行(由租约到期后重新领取)")
}
m.cancel()
}
// Reset 在 StopAll 后重新允许订阅(切回 carrier 模式时使用)。
func (m *ConsumerManager) Reset() {
m.mu.Lock()
defer m.mu.Unlock()
if m.ctx.Err() != nil {
m.ctx, m.cancel = context.WithCancel(context.Background())
}
}
// SplitAddresses 解析逗号分隔的地址列表。
func SplitAddresses(raw string) []string {
var result []string
for _, part := range strings.Split(raw, ",") {
if part = strings.TrimSpace(part); part != "" {
result = append(result, part)
}
}
return result
}
type nsqLogger struct {
logger *zap.Logger
}
// Output 实现 go-nsq 的日志接口。
func (l nsqLogger) Output(_ int, s string) error {
if l.logger != nil {
l.logger.Warn("NSQ", zap.String("message", s))
}
return nil
}

View File

@@ -101,7 +101,7 @@ func buildApprovalForm(ctx context.Context, businessType string, applicationID,
attachmentCount := 0
for _, mapping := range mappings {
raw, exists := lookupSnapshotValue(snapshot, mapping.BusinessField)
if !exists || raw == nil || isEmptyFileValue(mapping, raw) {
if !exists || raw == nil || isEmptyFileValue(mapping, raw) || isEmptyCustomerAccountInfo(mapping, raw) {
if requiredControls[mapping.ControlID] {
return approvalFormBuildResult{}, errors.New(errors.CodeInvalidStatus, "审批业务快照缺少模板必填字段: "+mapping.BusinessField)
}
@@ -142,6 +142,15 @@ func isEmptyFileValue(mapping dto.WeComControlMappingItem, raw any) bool {
}
}
// isEmptyCustomerAccountInfo 判断客户收款信息是否为空文本;只有 0 元退款可能为空,按未填写处理,由模板必填规则决定是否拦截。
func isEmptyCustomerAccountInfo(mapping dto.WeComControlMappingItem, raw any) bool {
if strings.TrimSpace(mapping.BusinessField) != constants.ApprovalFieldCustomerAccountInfo {
return false
}
value, ok := raw.(string)
return ok && strings.TrimSpace(value) == ""
}
// lookupSnapshotValue 支持用点分路径读取嵌套业务快照,避免模板映射绑定具体 DTO。
func lookupSnapshotValue(snapshot map[string]any, path string) (any, bool) {
segments := strings.Split(strings.TrimSpace(path), ".")

View File

@@ -0,0 +1,44 @@
package model
import "time"
// CardFlowDay 卡日流量快照:每卡每天一条本月自然月累计。
//
// MonthUsedMB 按通道上游流量重置日换算为自然月累计,同一天只增不减;SettledMB 是当天已结算的增量合计
// (发出的增量、无生效套餐时丢弃的增量、切换时视为老算法已结算的部分)。由观测链路原子写入,永久保留。
type CardFlowDay struct {
ID uint `gorm:"column:id;primaryKey" json:"id"`
IotCardID uint `gorm:"column:iot_card_id;type:bigint;not null;uniqueIndex:uk_card_flow_day_card_date" json:"iot_card_id"`
FlowDate time.Time `gorm:"column:flow_date;type:date;not null;uniqueIndex:uk_card_flow_day_card_date" json:"flow_date"`
MonthUsedMB float64 `gorm:"column:month_used_mb;type:numeric(14,3);not null;default:0" json:"month_used_mb"`
VendorUsedMB float64 `gorm:"column:vendor_used_mb;type:numeric(14,3);not null;default:0" json:"vendor_used_mb"`
SettledMB float64 `gorm:"column:settled_mb;type:numeric(14,3);not null;default:0" json:"settled_mb"`
StaleGuarded bool `gorm:"column:stale_guarded;not null;default:false" json:"stale_guarded"`
ObsCount int `gorm:"column:obs_count;not null;default:0" json:"obs_count"`
CreatedAt time.Time `gorm:"column:created_at;autoCreateTime" json:"created_at"`
UpdatedAt time.Time `gorm:"column:updated_at;autoUpdateTime" json:"updated_at"`
}
// TableName 指定表名
func (CardFlowDay) TableName() string {
return "tb_card_flow_day"
}
// CardFlowSwitch 卡流量计量切换水位:卡第一次走新算法时写入,退回老算法时由维护者删除。
type CardFlowSwitch struct {
ID uint `gorm:"column:id;primaryKey" json:"id"`
IotCardID uint `gorm:"column:iot_card_id;type:bigint;not null;uniqueIndex:uk_card_flow_switch_card" json:"iot_card_id"`
SwitchedAt time.Time `gorm:"column:switched_at;type:timestamptz;not null" json:"switched_at"`
MonthUsedMB float64 `gorm:"column:month_used_mb;type:numeric(14,3);not null;default:0" json:"month_used_mb"`
VendorUsedMB float64 `gorm:"column:vendor_used_mb;type:numeric(14,3);not null;default:0" json:"vendor_used_mb"`
OldBaselineMB float64 `gorm:"column:old_baseline_mb;type:numeric(14,3);not null;default:0" json:"old_baseline_mb"`
OldMonthUsageMB float64 `gorm:"column:old_month_usage_mb;type:numeric(14,3);not null;default:0" json:"old_month_usage_mb"`
OldReadingAccepted bool `gorm:"column:old_reading_accepted;not null;default:false" json:"old_reading_accepted"`
OldIncrementMB float64 `gorm:"column:old_increment_mb;type:numeric(14,3);not null;default:0" json:"old_increment_mb"`
CreatedAt time.Time `gorm:"column:created_at;autoCreateTime" json:"created_at"`
}
// TableName 指定表名
func (CardFlowSwitch) TableName() string {
return "tb_card_flow_switch"
}

View File

@@ -22,6 +22,9 @@ type Carrier struct {
TrafficThresholdValue *float64 `gorm:"column:traffic_threshold_value;type:numeric(18,2);comment:通道流量阈值数值,单位由 traffic_threshold_unit 决定" json:"traffic_threshold_value"`
// TrafficThresholdUnit 是阈值单位,取值 MB/GB(1 GB = 1024 MB),换算后与网关累计读数同单位比较;空字符串表示未配置。
TrafficThresholdUnit string `gorm:"column:traffic_threshold_unit;type:varchar(8);not null;default:'';comment:通道流量阈值单位 MB/GB,空字符串-未配置" json:"traffic_threshold_unit"`
// RealnameControlEnabled 是通道实名管控开关,取值 0-不管控(默认)1-管控;只用于停复机,
// 且只在停复机新规则总开关开启后参与判断,不复用 RealnameLinkType 等现有实名字段。
RealnameControlEnabled int `gorm:"column:realname_control_enabled;type:smallint;not null;default:0;comment:通道实名管控 0-不管控 1-管控" json:"realname_control_enabled"`
}
// TableName 指定表名

View File

@@ -0,0 +1,76 @@
package model
import (
"time"
"gorm.io/datatypes"
"gorm.io/gorm"
)
// CarrierPollingParam 运营商账户调度参数;未配置的账户没有记录,使用全局默认。
type CarrierPollingParam struct {
ID uint `gorm:"column:id;primaryKey" json:"id"`
CarrierID uint `gorm:"column:carrier_id;not null" json:"carrier_id"`
InflightLimit *int `gorm:"column:inflight_limit" json:"inflight_limit"`
CardMaxInFlight *int `gorm:"column:card_max_in_flight" json:"card_max_in_flight"`
StopResumeMaxInFlight *int `gorm:"column:stop_resume_max_in_flight" json:"stop_resume_max_in_flight"`
SRInflightLimit *int `gorm:"column:sr_inflight_limit" json:"sr_inflight_limit"`
PriorityMaxRatio *float64 `gorm:"column:priority_max_ratio" json:"priority_max_ratio"`
StopResumeMinInterval *int `gorm:"column:stop_resume_min_interval" json:"stop_resume_min_interval"`
StopResumeConfirmDelay *int `gorm:"column:stop_resume_confirm_delay" json:"stop_resume_confirm_delay"`
Creator uint `gorm:"column:creator;not null;default:0" json:"creator"`
Updater uint `gorm:"column:updater;not null;default:0" json:"updater"`
CreatedAt time.Time `gorm:"column:created_at" json:"created_at"`
UpdatedAt time.Time `gorm:"column:updated_at" json:"updated_at"`
DeletedAt gorm.DeletedAt `gorm:"column:deleted_at;index" json:"-"`
}
// TableName 返回运营商账户调度参数表名。
func (CarrierPollingParam) TableName() string {
return "tb_carrier_polling_param"
}
// PollingCheckScene 业务事件与读入口请求检查场景;场景由代码预置,后台只可修改检查类型、确认延迟与冷却。
type PollingCheckScene struct {
ID uint `gorm:"column:id;primaryKey" json:"id"`
Scene string `gorm:"column:scene;type:varchar(50);not null" json:"scene"`
SceneName string `gorm:"column:scene_name;type:varchar(100);not null" json:"scene_name"`
CheckTypes datatypes.JSON `gorm:"column:check_types;type:jsonb;not null" json:"check_types"`
ConfirmDelays datatypes.JSON `gorm:"column:confirm_delays;type:jsonb;not null" json:"confirm_delays"`
Cooldown *int `gorm:"column:cooldown" json:"cooldown"`
Updater uint `gorm:"column:updater;not null;default:0" json:"updater"`
CreatedAt time.Time `gorm:"column:created_at" json:"created_at"`
UpdatedAt time.Time `gorm:"column:updated_at" json:"updated_at"`
}
// TableName 返回请求检查场景表名。
func (PollingCheckScene) TableName() string {
return "tb_polling_check_scene"
}
// PollingFailureAlert 卡轮询失败告警;同一卡同一检查类型只保留一条未处理告警。
type PollingFailureAlert struct {
ID uint `gorm:"column:id;primaryKey" json:"id"`
AlertLevel string `gorm:"column:alert_level;type:varchar(20);not null;default:card" json:"alert_level"`
CardID uint `gorm:"column:card_id;not null;default:0" json:"card_id"`
ICCID string `gorm:"column:iccid;type:varchar(30);not null;default:''" json:"iccid"`
CarrierID uint `gorm:"column:carrier_id;not null;default:0" json:"carrier_id"`
CheckType string `gorm:"column:check_type;type:varchar(30);not null" json:"check_type"`
ErrorCategory string `gorm:"column:error_category;type:varchar(50);not null;default:''" json:"error_category"`
FailCount int `gorm:"column:fail_count;not null;default:0" json:"fail_count"`
FirstFailedAt *time.Time `gorm:"column:first_failed_at" json:"first_failed_at"`
LastFailedAt *time.Time `gorm:"column:last_failed_at" json:"last_failed_at"`
LastError string `gorm:"column:last_error;type:varchar(500);not null;default:''" json:"last_error"`
Status int `gorm:"column:status;type:smallint;not null;default:1" json:"status"`
HandledBy uint `gorm:"column:handled_by;not null;default:0" json:"handled_by"`
HandledAt *time.Time `gorm:"column:handled_at" json:"handled_at"`
HandleRemark string `gorm:"column:handle_remark;type:varchar(500);not null;default:''" json:"handle_remark"`
RecoveredAt *time.Time `gorm:"column:recovered_at" json:"recovered_at"`
CreatedAt time.Time `gorm:"column:created_at" json:"created_at"`
UpdatedAt time.Time `gorm:"column:updated_at" json:"updated_at"`
}
// TableName 返回卡轮询失败告警表名。
func (PollingFailureAlert) TableName() string {
return "tb_polling_failure_alert"
}

View File

@@ -35,7 +35,7 @@ type AssetResolveResponse struct {
VirtualUsedMB float64 `json:"virtual_used_mb" description:"当前主套餐展示已用量(MB)"`
ReductionPct float64 `json:"reduction_pct" description:"展示增幅比例,公式为(real_total_mb / virtual_total_mb) - 1"`
EnableVirtualData bool `json:"enable_virtual_data" description:"当前主套餐是否启用虚流量(按套餐使用记录快照返回)"`
DeviceProtectStatus string `json:"device_protect_status,omitempty" description:"设备保护期状态:none/stop/start(仅asset_type=device时有效)"`
DeviceProtectStatus string `json:"device_protect_status,omitempty" deprecated:"true" description:"已废弃:设备保护期已下线,固定返回 none,仅为兼容保留(仅asset_type=device时有效)"`
// 绑定关系字段
AssociatedPhones []string `json:"associated_phones" description:"当前关联手机号列表(按资产数据范围返回完整手机号,无关联时为空数组)"`
ICCID string `json:"iccid,omitempty" description:"卡ICCID(asset_type=card时有效)"`
@@ -141,7 +141,7 @@ type AssetRealtimeStatusResponse struct {
LastDataCheckAt *time.Time `json:"last_data_check_at" description:"最后一次流量检查时间(asset_type=card时有效)"`
LastRealNameCheckAt *time.Time `json:"last_real_name_check_at" description:"最后一次实名检查时间(asset_type=card时有效)"`
LastCardStatusCheckAt *time.Time `json:"last_card_status_check_at" description:"最后一次卡状态检查时间(asset_type=card时有效)"`
DeviceProtectStatus string `json:"device_protect_status,omitempty" description:"保护期状态(asset_type=device时有效):none/stop/start"`
DeviceProtectStatus string `json:"device_protect_status,omitempty" deprecated:"true" description:"已废弃:设备保护期已下线,固定返回 none,仅为兼容保留(asset_type=device时有效)"`
Cards []BoundCardInfo `json:"cards,omitempty" description:"绑定卡状态列表(asset_type=device时有效)"`
DeviceRealtime *DeviceGatewayInfo `json:"device_realtime" description:"设备实时状态(来自 Gateway sync-info,始终返回,失败时仅含 gateway_msg)"`
}

View File

@@ -13,15 +13,17 @@ type CreateCarrierRequest struct {
TrafficThresholdEnabled *int `json:"traffic_threshold_enabled" validate:"omitempty,oneof=0 1" description:"通道流量阈值是否启用 (0:禁用, 1:启用),仅超级管理员与平台账号可配置"`
TrafficThresholdValue *float64 `json:"traffic_threshold_value" validate:"omitempty,gt=0" description:"通道流量阈值数值,必须为正数,单位由 traffic_threshold_unit 决定,仅超级管理员与平台账号可配置"`
TrafficThresholdUnit *string `json:"traffic_threshold_unit" validate:"omitempty,oneof=MB GB" description:"通道流量阈值单位 (MB:兆字节, GB:吉字节,1 GB = 1024 MB),仅超级管理员与平台账号可配置"`
RealnameControlEnabled *int `json:"realname_control_enabled" validate:"omitempty,oneof=0 1" description:"通道实名管控 (0:不管控, 1:管控),默认0;只在停复机新规则开启后影响停复机:管控时未实名卡停机、实名后才复机。仅超级管理员与平台账号可配置"`
}
// HasTrafficThresholdFields 判断请求是否携带通道流量阈值字段。
// 非超管、非平台账号提交携带阈值字段的请求必须被整体拒绝。
// HasTrafficThresholdFields 判断请求是否携带仅平台账号可配置的通道字段(通道流量阈值、通道实名管控)。
// 非超管、非平台账号提交携带这些字段的请求必须被整体拒绝。
func (r *CreateCarrierRequest) HasTrafficThresholdFields() bool {
if r == nil {
return false
}
return r.TrafficThresholdEnabled != nil || r.TrafficThresholdValue != nil || r.TrafficThresholdUnit != nil
return r.TrafficThresholdEnabled != nil || r.TrafficThresholdValue != nil || r.TrafficThresholdUnit != nil ||
r.RealnameControlEnabled != nil
}
type UpdateCarrierRequest struct {
@@ -33,15 +35,18 @@ type UpdateCarrierRequest struct {
TrafficThresholdEnabled *int `json:"traffic_threshold_enabled" validate:"omitempty,oneof=0 1" description:"通道流量阈值是否启用 (0:禁用, 1:启用),仅超级管理员与平台账号可配置"`
TrafficThresholdValue *float64 `json:"traffic_threshold_value" validate:"omitempty,gt=0" description:"通道流量阈值数值,必须为正数,单位由 traffic_threshold_unit 决定,仅超级管理员与平台账号可配置"`
TrafficThresholdUnit *string `json:"traffic_threshold_unit" validate:"omitempty,oneof=MB GB" description:"通道流量阈值单位 (MB:兆字节, GB:吉字节,1 GB = 1024 MB),仅超级管理员与平台账号可配置"`
RealnameControlEnabled *int `json:"realname_control_enabled" validate:"omitempty,oneof=0 1" description:"通道实名管控 (0:不管控, 1:管控),默认0;只在停复机新规则开启后影响停复机:管控时未实名卡停机、实名后才复机。仅超级管理员与平台账号可配置"`
RealnameControlConfirm *bool `json:"realname_control_confirm" description:"把实名管控从 0 改为 1 时必须传 true,确认该通道未实名的开机卡会在下一次评估时被停机"`
}
// HasTrafficThresholdFields 判断请求是否携带通道流量阈值字段。
// 非超管、非平台账号提交携带阈值字段的请求必须被整体拒绝。
// HasTrafficThresholdFields 判断请求是否携带仅平台账号可配置的通道字段(通道流量阈值、通道实名管控)。
// 非超管、非平台账号提交携带这些字段的请求必须被整体拒绝。
func (r *UpdateCarrierRequest) HasTrafficThresholdFields() bool {
if r == nil {
return false
}
return r.TrafficThresholdEnabled != nil || r.TrafficThresholdValue != nil || r.TrafficThresholdUnit != nil
return r.TrafficThresholdEnabled != nil || r.TrafficThresholdValue != nil || r.TrafficThresholdUnit != nil ||
r.RealnameControlEnabled != nil
}
type CarrierListRequest struct {
@@ -70,6 +75,7 @@ type CarrierResponse struct {
TrafficThresholdEnabled *int `json:"traffic_threshold_enabled,omitempty" description:"通道流量阈值是否启用 (0:禁用, 1:启用),仅超级管理员与平台账号可见"`
TrafficThresholdValue *float64 `json:"traffic_threshold_value,omitempty" description:"通道流量阈值数值,单位由 traffic_threshold_unit 决定,未配置时不返回,仅超级管理员与平台账号可见"`
TrafficThresholdUnit *string `json:"traffic_threshold_unit,omitempty" description:"通道流量阈值单位 (MB:兆字节, GB:吉字节,1 GB = 1024 MB),未配置时不返回,仅超级管理员与平台账号可见"`
RealnameControlEnabled *int `json:"realname_control_enabled,omitempty" description:"通道实名管控 (0:不管控, 1:管控),仅超级管理员与平台账号可见"`
CreatedAt string `json:"created_at" description:"创建时间"`
UpdatedAt string `json:"updated_at" description:"更新时间"`
}

View File

@@ -0,0 +1,185 @@
package dto
import "time"
// CarrierPollingParamItem 运营商账户调度参数(含全局默认合并后的生效值)。
type CarrierPollingParamItem struct {
CarrierID uint `json:"carrier_id" description:"运营商账户ID"`
CarrierName string `json:"carrier_name" description:"运营商账户名称"`
IsDefault bool `json:"is_default" description:"是否未单独配置、全部使用全局默认"`
InflightLimit int `json:"inflight_limit" description:"生效的执行上限(跨 Worker 实例同时执行卡任务数)"`
CardMaxInFlight int `json:"card_max_in_flight" description:"生效的每进程卡任务 max_in_flight"`
PriorityMaxRatio *float64 `json:"priority_max_ratio" description:"优先队列每轮最大占比(0~1),空表示不限"`
StopResumeMaxInFlight *int `json:"stop_resume_max_in_flight" description:"每进程停复机 max_in_flight(停复机指令队列使用),空表示默认"`
SRInflightLimit *int `json:"sr_inflight_limit" description:"停复机执行上限(停复机指令队列使用),空表示默认"`
StopResumeMinInterval *int `json:"stop_resume_min_interval" description:"单卡两次停复机调用最小间隔(秒),空表示默认"`
StopResumeConfirmDelay *int `json:"stop_resume_confirm_delay" description:"停复机结果未知后确认延迟(秒),空表示默认"`
ConfiguredInflight *int `json:"configured_inflight_limit" description:"单独配置的执行上限,空表示使用默认"`
ConfiguredMaxInFlight *int `json:"configured_card_max_in_flight" description:"单独配置的每进程 max_in_flight,空表示使用默认"`
ConsumeCapacity int `json:"consume_capacity" description:"当前订阅进程 max_in_flight 之和(执行上限不应超过该值)"`
}
// CarrierPollingParamListResponse 运营商账户调度参数列表。
type CarrierPollingParamListResponse struct {
Items []CarrierPollingParamItem `json:"items" description:"各运营商账户调度参数"`
SubscriberProcesses int `json:"subscriber_processes" description:"订阅卡任务的 Worker 进程数"`
DefaultInflight int `json:"default_inflight_limit" description:"全局默认执行上限"`
DefaultMaxInFlight int `json:"default_card_max_in_flight" description:"全局默认每进程 max_in_flight"`
}
// CarrierPollingParamPath 运营商账户路径参数。
type CarrierPollingParamPath struct {
CarrierID uint `json:"-" path:"carrier_id" required:"true" description:"运营商账户ID"`
}
// SaveCarrierPollingParamRequest 保存运营商账户调度参数;字段为空表示该项使用全局默认。
type SaveCarrierPollingParamRequest struct {
CarrierID uint `json:"-" path:"carrier_id" required:"true" description:"运营商账户ID"`
InflightLimit *int `json:"inflight_limit" validate:"omitempty,min=1,max=10000" minimum:"1" maximum:"10000" description:"执行上限,空表示默认"`
CardMaxInFlight *int `json:"card_max_in_flight" validate:"omitempty,min=1,max=2000" minimum:"1" maximum:"2000" description:"每进程卡任务 max_in_flight,空表示默认"`
StopResumeMaxInFlight *int `json:"stop_resume_max_in_flight" validate:"omitempty,min=1,max=1000" minimum:"1" maximum:"1000" description:"每进程停复机 max_in_flight,空表示默认"`
SRInflightLimit *int `json:"sr_inflight_limit" validate:"omitempty,min=1,max=10000" minimum:"1" maximum:"10000" description:"停复机执行上限,空表示默认"`
PriorityMaxRatio *float64 `json:"priority_max_ratio" validate:"omitempty,gt=0,lte=1" description:"优先队列每轮最大占比(0~1],空表示不限"`
StopResumeMinInterval *int `json:"stop_resume_min_interval" validate:"omitempty,min=1,max=86400" minimum:"1" maximum:"86400" description:"单卡停复机最小间隔(秒),空表示默认"`
StopResumeConfirmDelay *int `json:"stop_resume_confirm_delay" validate:"omitempty,min=1,max=86400" minimum:"1" maximum:"86400" description:"停复机结果未知确认延迟(秒),空表示默认"`
}
// PollingCheckSceneItem 请求检查场景。
type PollingCheckSceneItem struct {
Scene string `json:"scene" description:"场景编码 (order_paid:订单支付, package_activated:套餐激活, stop_resume_succeeded:停复机成功, device_operation:设备操作, auto_renewal_purchase:自动续费购包, client_asset_detail:C 端资产详情, admin_asset_realtime:后台资产实时状态, realname_link:实名链接)"`
SceneName string `json:"scene_name" description:"场景名称"`
CheckTypes []string `json:"check_types" description:"检查类型 (card_status:卡状态, carddata:流量, realname:实名)"`
ConfirmDelays []int `json:"confirm_delays" description:"延迟确认序列(秒),空表示只立即检查一次"`
Cooldown *int `json:"cooldown" description:"冷却时长(秒),最近一次成功检查未超过该时长则不请求;空表示不冷却"`
UpdatedAt time.Time `json:"updated_at" description:"更新时间"`
}
// UpdatePollingCheckSceneRequest 修改请求检查场景(场景由代码预置,只可改检查类型、确认延迟与冷却)。
type UpdatePollingCheckSceneRequest struct {
Scene string `json:"-" path:"scene" required:"true" description:"场景编码"`
CheckTypes []string `json:"check_types" validate:"required,min=1,max=3,dive,oneof=card_status carddata realname" required:"true" description:"检查类型 (card_status:卡状态, carddata:流量, realname:实名)"`
ConfirmDelays []int `json:"confirm_delays" validate:"omitempty,max=5,dive,min=1,max=86400" description:"延迟确认序列(秒,递增),空表示只立即检查一次"`
Cooldown *int `json:"cooldown" validate:"omitempty,min=1,max=86400" minimum:"1" maximum:"86400" description:"冷却时长(秒),空表示不冷却"`
}
// PollingFailureAlertListRequest 卡轮询失败告警列表请求。
type PollingFailureAlertListRequest struct {
Page int `json:"page" query:"page" validate:"omitempty,min=1" minimum:"1" description:"页码"`
PageSize int `json:"page_size" query:"page_size" validate:"omitempty,min=1,max=100" minimum:"1" maximum:"100" description:"每页数量"`
CarrierID *uint `json:"carrier_id" query:"carrier_id" validate:"omitempty,min=1" description:"运营商账户ID"`
CheckType string `json:"check_type" query:"check_type" validate:"omitempty,oneof=card_status carddata realname package all" description:"检查类型 (card_status:卡状态, carddata:流量, realname:实名, package:套餐, all:账户级)"`
ErrorCategory string `json:"error_category" query:"error_category" validate:"omitempty,max=50" description:"错误类别"`
Status *int `json:"status" query:"status" validate:"omitempty,oneof=1 2 3 4" description:"状态 (1:未处理, 2:已恢复, 3:已关闭轮询, 4:已处理)"`
AlertLevel string `json:"alert_level" query:"alert_level" validate:"omitempty,oneof=card carrier" description:"告警级别 (card:卡级, carrier:账户级)"`
ICCID string `json:"iccid" query:"iccid" validate:"omitempty,max=30" description:"ICCID(精确)"`
StartTime string `json:"start_time" query:"start_time" description:"最近失败时间起始(带时区的 RFC3339 秒级时间)"`
EndTime string `json:"end_time" query:"end_time" description:"最近失败时间结束(带时区的 RFC3339 秒级时间)"`
}
// PollingFailureAlertItem 卡轮询失败告警。
type PollingFailureAlertItem struct {
ID uint `json:"id" description:"告警ID"`
AlertLevel string `json:"alert_level" description:"告警级别 (card:卡级, carrier:账户级)"`
CardID uint `json:"card_id" description:"卡ID,账户级为 0"`
ICCID string `json:"iccid" description:"ICCID"`
CarrierID uint `json:"carrier_id" description:"运营商账户ID"`
CarrierName string `json:"carrier_name" description:"运营商账户名称"`
CheckType string `json:"check_type" description:"检查类型"`
CheckTypeName string `json:"check_type_name" description:"检查类型名称"`
ErrorCategory string `json:"error_category" description:"最近错误类别"`
FailCount int `json:"fail_count" description:"连续失败次数;账户级为窗口内告警数"`
FirstFailedAt *time.Time `json:"first_failed_at" description:"首次失败时间"`
LastFailedAt *time.Time `json:"last_failed_at" description:"最近失败时间"`
LastError string `json:"last_error" description:"最近错误"`
Status int `json:"status" description:"状态 (1:未处理, 2:已恢复, 3:已关闭轮询, 4:已处理)"`
StatusName string `json:"status_name" description:"状态名称"`
HandledBy uint `json:"handled_by" description:"处理人"`
HandledAt *time.Time `json:"handled_at" description:"处理时间"`
HandleRemark string `json:"handle_remark" description:"处理备注"`
RecoveredAt *time.Time `json:"recovered_at" description:"自动恢复时间"`
CreatedAt time.Time `json:"created_at" description:"创建时间"`
}
// PollingFailureAlertPageResult 卡轮询失败告警分页结果。
type PollingFailureAlertPageResult struct {
List []PollingFailureAlertItem `json:"items" description:"告警列表"`
Total int64 `json:"total" description:"总数"`
Page int `json:"page" description:"当前页"`
PageSize int `json:"size" description:"每页数量"`
}
// PollingFailureAlertSummaryItem 未处理告警按运营商账户 × 错误类别汇总。
type PollingFailureAlertSummaryItem struct {
CarrierID uint `json:"carrier_id" description:"运营商账户ID"`
CarrierName string `json:"carrier_name" description:"运营商账户名称"`
ErrorCategory string `json:"error_category" description:"错误类别"`
OpenCount int64 `json:"open_count" description:"未处理卡级告警数"`
}
// PollingFailureAlertIDPath 告警路径参数。
type PollingFailureAlertIDPath struct {
ID uint `json:"-" path:"id" required:"true" description:"告警ID"`
}
// HandlePollingFailureAlertRequest 标记告警已处理。
type HandlePollingFailureAlertRequest struct {
ID uint `json:"-" path:"id" required:"true" description:"告警ID"`
Status int `json:"status" validate:"required,oneof=4" required:"true" description:"目标状态 (4:已处理);关闭轮询请使用资产轮询开关接口"`
Remark string `json:"remark" validate:"omitempty,max=500" maxLength:"500" description:"处理备注"`
}
// CardPollingStatePath 卡ID路径参数。
type CardPollingStatePath struct {
ID uint `json:"-" path:"id" required:"true" description:"卡ID"`
}
// CardPollingTypeState 卡某检查类型的轮询状态。
type CardPollingTypeState struct {
CheckType string `json:"check_type" description:"检查类型"`
CheckTypeName string `json:"check_type_name" description:"检查类型名称"`
NextDueAt *time.Time `json:"next_due_at" description:"下次到期时间"`
LastSuccessAt *time.Time `json:"last_success_at" description:"最近一次成功时间"`
FailCount int `json:"fail_count" description:"连续失败次数"`
FirstFailedAt *time.Time `json:"first_failed_at" description:"首次失败时间"`
LastError string `json:"last_error" description:"最近错误(类别|信息)"`
PendingConfirm []int `json:"pending_confirm" description:"待执行的延迟确认(秒)"`
}
// CardPollingStateResponse 单卡轮询状态(只读)。
type CardPollingStateResponse struct {
CardID uint `json:"card_id" description:"卡ID"`
ICCID string `json:"iccid" description:"ICCID"`
CarrierID uint `json:"carrier_id" description:"运营商账户ID"`
SchedulerMode string `json:"scheduler_mode" description:"当前调度模式 (legacy:旧分片调度, carrier:按运营商账户调度)"`
EnablePolling bool `json:"enable_polling" description:"卡轮询开关"`
Queue string `json:"queue" description:"所在队列 (priority:优先, queue:普通, none:不在轮询中)"`
QueueScoreAt *time.Time `json:"queue_score_at" description:"队列中下次可领取时间"`
Running bool `json:"running" description:"是否执行中"`
ClaimExpireAt *time.Time `json:"claim_expire_at" description:"处理中标记到期时间"`
PriorityMark bool `json:"priority_mark" description:"执行中收到请求检查,收尾后进入优先队列"`
Types []CardPollingTypeState `json:"types" description:"各检查类型状态"`
}
// CarrierPollingStatsItem 按运营商账户的轮询监控。
type CarrierPollingStatsItem struct {
CarrierID uint `json:"carrier_id" description:"运营商账户ID"`
CarrierName string `json:"carrier_name" description:"运营商账户名称"`
PriorityLength int64 `json:"priority_length" description:"优先队列长度"`
QueueLength int64 `json:"queue_length" description:"普通队列长度"`
DueCount int64 `json:"due_count" description:"已到期卡数(优先 + 普通)"`
Running int64 `json:"running" description:"执行中卡任务数"`
InflightLimit int `json:"inflight_limit" description:"执行上限"`
QPSLastMinute float64 `json:"qps_last_minute" description:"上一分钟实际网关调用 QPS(查询类)"`
RateLimitWaits int64 `json:"rate_limit_waits" description:"上一分钟限流等待次数"`
RateLimited int64 `json:"rate_limited" description:"上一分钟限流拒绝或等待超时次数"`
ExpiredClaims1h int64 `json:"expired_claims_1h" description:"近 1 小时处理中标记到期次数(执行崩溃或超时)"`
OpenAlerts int64 `json:"open_alerts" description:"未处理卡级失败告警数"`
OpenAlertsByType map[string]int64 `json:"open_alerts_by_category" description:"未处理卡级失败告警数(按错误类别)"`
}
// CarrierPollingStatsResponse 按运营商账户的轮询监控。
type CarrierPollingStatsResponse struct {
SchedulerMode string `json:"scheduler_mode" description:"当前调度模式"`
InitState string `json:"init_state" description:"新结构初始化状态 (running / done,空表示未初始化)"`
Items []CarrierPollingStatsItem `json:"items" description:"各运营商账户监控"`
}

View File

@@ -72,7 +72,7 @@ type AssetInfoResponse struct {
MaxSimSlots int `json:"max_sim_slots,omitempty" description:"最大插槽数"`
BoundCardCount int `json:"bound_card_count,omitempty" description:"绑定卡数量"`
Cards []BoundCardInfo `json:"cards,omitempty" description:"绑定卡列表(含每张卡的ICCID/MSISDN/网络状态/实名状态/插槽位置)"`
DeviceProtectStatus string `json:"device_protect_status,omitempty" description:"设备保护期状态(none:无, stop:停机保护, start:开机保护)"`
DeviceProtectStatus string `json:"device_protect_status,omitempty" deprecated:"true" description:"已废弃:设备保护期已下线,固定返回 none,仅为兼容保留"`
// === 卡专属字段(asset_type=card 时有效) ===
ICCID string `json:"iccid,omitempty" description:"卡ICCID"`

View File

@@ -4,7 +4,7 @@ import "time"
// CreateExportTaskRequest 创建导出任务请求。
type CreateExportTaskRequest struct {
Scene string `json:"scene" validate:"required,oneof=device iot_card order package agent_wallet_transaction agent_recharge refund exchange commission_record package_traffic_alert expiring_asset operations_activation operations_renewal" required:"true" description:"导出场景 (device:设备, iot_card:IoT卡, order:订单, package:套餐, agent_wallet_transaction:代理主钱包流水, agent_recharge:代理充值, refund:退款, exchange:换货, commission_record:佣金明细, package_traffic_alert:套餐真流量达量预警, expiring_asset:临期资产, operations_activation:设备激活情况报表, operations_renewal:套餐续费情况报表)"`
Scene string `json:"scene" validate:"required,oneof=device iot_card order package agent_wallet_transaction agent_recharge refund exchange commission_record package_traffic_alert expiring_asset operations_activation operations_renewal stop_resume_rule_dry_run" required:"true" description:"导出场景 (device:设备, iot_card:IoT卡, order:订单, package:套餐, agent_wallet_transaction:代理主钱包流水, agent_recharge:代理充值, refund:退款, exchange:换货, commission_record:佣金明细, package_traffic_alert:套餐真流量达量预警, expiring_asset:临期资产, operations_activation:设备激活情况报表, operations_renewal:套餐续费情况报表, stop_resume_rule_dry_run:停复机规则演练明细)"`
Format string `json:"format" validate:"required,oneof=xlsx csv" required:"true" description:"导出格式 (xlsx:Excel, csv:CSV)"`
Query map[string]interface{} `json:"query,omitempty" description:"导出筛选参数(JSON对象,可选);时间筛选固定使用 filters.start_time 与 filters.end_time,取值必须为带显式时区的 RFC3339 秒级时间"`
}
@@ -22,7 +22,7 @@ type CreateExportTaskResponse struct {
type ListExportTaskRequest struct {
Page int `json:"page" query:"page" validate:"omitempty,min=1" minimum:"1" description:"页码"`
PageSize int `json:"page_size" query:"page_size" validate:"omitempty,min=1,max=100" minimum:"1" maximum:"100" description:"每页数量"`
Scene string `json:"scene" query:"scene" validate:"omitempty,oneof=device iot_card order package agent_wallet_transaction agent_recharge refund exchange commission_record package_traffic_alert expiring_asset operations_activation operations_renewal" description:"导出场景 (device:设备, iot_card:IoT卡, order:订单, package:套餐, agent_wallet_transaction:代理主钱包流水, agent_recharge:代理充值, refund:退款, exchange:换货, commission_record:佣金明细, package_traffic_alert:套餐真流量达量预警, expiring_asset:临期资产, operations_activation:设备激活情况报表, operations_renewal:套餐续费情况报表)"`
Scene string `json:"scene" query:"scene" validate:"omitempty,oneof=device iot_card order package agent_wallet_transaction agent_recharge refund exchange commission_record package_traffic_alert expiring_asset operations_activation operations_renewal stop_resume_rule_dry_run" description:"导出场景 (device:设备, iot_card:IoT卡, order:订单, package:套餐, agent_wallet_transaction:代理主钱包流水, agent_recharge:代理充值, refund:退款, exchange:换货, commission_record:佣金明细, package_traffic_alert:套餐真流量达量预警, expiring_asset:临期资产, operations_activation:设备激活情况报表, operations_renewal:套餐续费情况报表, stop_resume_rule_dry_run:停复机规则演练明细)"`
Status *int `json:"status" query:"status" validate:"omitempty,min=1,max=5" minimum:"1" maximum:"5" description:"任务状态 (1:待处理, 2:处理中, 3:已完成, 4:已失败, 5:已取消)"`
StartTime string `json:"start_time" query:"start_time" description:"创建时间起始(带时区的 RFC3339 秒级时间,闭区间含该时刻,如 2026-09-01T00:00:00+08:00)"`
EndTime string `json:"end_time" query:"end_time" description:"创建时间结束(带时区的 RFC3339 秒级时间,闭区间含该时刻,如 2026-09-30T23:59:59+08:00)"`
@@ -33,7 +33,7 @@ type ExportTaskItem struct {
ID uint `json:"id" description:"任务ID"`
TaskID uint `json:"task_id" description:"任务ID"`
TaskNo string `json:"task_no" description:"任务编号"`
Scene string `json:"scene" description:"导出场景 (device:设备, iot_card:IoT卡, order:订单, package:套餐, agent_wallet_transaction:代理主钱包流水, agent_recharge:代理充值, refund:退款, exchange:换货, commission_record:佣金明细, package_traffic_alert:套餐真流量达量预警, operations_activation:设备激活情况报表, operations_renewal:套餐续费情况报表)"`
Scene string `json:"scene" description:"导出场景 (device:设备, iot_card:IoT卡, order:订单, package:套餐, agent_wallet_transaction:代理主钱包流水, agent_recharge:代理充值, refund:退款, exchange:换货, commission_record:佣金明细, package_traffic_alert:套餐真流量达量预警, operations_activation:设备激活情况报表, operations_renewal:套餐续费情况报表, stop_resume_rule_dry_run:停复机规则演练明细)"`
Format string `json:"format" description:"导出格式 (xlsx:Excel, csv:CSV)"`
Status int `json:"status" description:"任务状态 (1:待处理, 2:处理中, 3:已完成, 4:已失败, 5:已取消)"`
StatusName string `json:"status_name" description:"任务状态名称(中文)"`

View File

@@ -0,0 +1,273 @@
package dto
import "time"
// RateLimitGroupListRequest 网关限流组列表请求
type RateLimitGroupListRequest struct {
Page int `json:"page" query:"page" validate:"omitempty,min=1" minimum:"1" description:"页码"`
PageSize int `json:"page_size" query:"page_size" validate:"omitempty,min=1,max=100" minimum:"1" maximum:"100" description:"每页数量"`
CarrierID *uint `json:"carrier_id" query:"carrier_id" validate:"omitempty" description:"运营商账户ID筛选"`
OnlyGlobal bool `json:"only_global" query:"only_global" description:"只看全局组"`
Status *int `json:"status" query:"status" validate:"omitempty,oneof=0 1" enum:"0,1" description:"状态 (0:停用, 1:启用)"`
}
// GatewayOperation 卡网关操作(query_flow:查流量, query_status:查状态, query_realname:查实名, stop:停机, start:复机)
type GatewayOperation string
// Enum 返回卡网关操作的全部取值,供接口文档生成数组元素枚举。
func (GatewayOperation) Enum() []interface{} {
return []interface{}{"query_flow", "query_status", "query_realname", "stop", "start"}
}
// SaveRateLimitGroupRequest 新建或修改网关限流组请求
type SaveRateLimitGroupRequest struct {
ID uint `json:"-" description:"限流组ID"`
CarrierID *uint `json:"carrier_id" validate:"omitempty" description:"运营商账户ID,为空表示全局组"`
GroupCode string `json:"group_code" validate:"required,max=50" required:"true" maxLength:"50" description:"组标识,同一账户内唯一"`
GroupName string `json:"group_name" validate:"required,max=100" required:"true" maxLength:"100" description:"组名称"`
MaxQPS int `json:"max_qps" validate:"required,min=1,max=10000" required:"true" minimum:"1" maximum:"10000" description:"每秒请求上限"`
Operations []GatewayOperation `json:"operations" validate:"required,min=1,dive,oneof=query_flow query_status query_realname stop start" required:"true" minItems:"1" description:"组内操作 (query_flow:查流量, query_status:查状态, query_realname:查实名, stop:停机, start:复机)"`
HighPriorityOperations []GatewayOperation `json:"high_priority_operations" validate:"omitempty,dive,oneof=query_flow query_status query_realname stop start" description:"高优先级操作,必须属于组内操作,设置预留额度时必填 (query_flow:查流量, query_status:查状态, query_realname:查实名, stop:停机, start:复机)"`
ReservedQPS int `json:"reserved_qps" validate:"min=0" minimum:"0" description:"预留给高优先级操作的额度,须小于每秒请求上限"`
Status *int `json:"status" validate:"omitempty,oneof=0 1" enum:"0,1" description:"状态 (0:停用, 1:启用),新建默认启用"`
}
// UpdateRateLimitGroupStatusRequest 启停网关限流组请求
type UpdateRateLimitGroupStatusRequest struct {
ID uint `json:"-" path:"id" description:"限流组ID"`
Status *int `json:"status" validate:"required,oneof=0 1" required:"true" enum:"0,1" description:"状态 (0:停用, 1:启用);停用后组内操作回落为默认每秒请求上限"`
}
// RateLimitGroupResponse 网关限流组响应
type RateLimitGroupResponse struct {
ID uint `json:"id" description:"限流组ID"`
CarrierID *uint `json:"carrier_id" description:"运营商账户ID,为空表示全局组"`
CarrierName string `json:"carrier_name" description:"运营商账户名称,全局组为空"`
GroupCode string `json:"group_code" description:"组标识"`
GroupName string `json:"group_name" description:"组名称"`
MaxQPS int `json:"max_qps" description:"每秒请求上限"`
Operations []string `json:"operations" description:"组内操作 (query_flow:查流量, query_status:查状态, query_realname:查实名, stop:停机, start:复机)"`
HighPriorityOperations []string `json:"high_priority_operations" description:"高优先级操作"`
ReservedQPS int `json:"reserved_qps" description:"预留给高优先级操作的额度"`
Status int `json:"status" description:"状态 (0:停用, 1:启用)"`
StatusName string `json:"status_name" description:"状态名称"`
CreatedAt time.Time `json:"created_at" description:"创建时间"`
UpdatedAt time.Time `json:"updated_at" description:"更新时间"`
}
// RateLimitGroupPageResult 网关限流组分页结果
type RateLimitGroupPageResult struct {
List []RateLimitGroupResponse `json:"items" description:"限流组列表"`
Total int64 `json:"total" description:"总数"`
Page int `json:"page" description:"当前页"`
PageSize int `json:"size" description:"每页数量"`
}
// GatewayErrorRuleListRequest 上游错误分类规则列表请求
type GatewayErrorRuleListRequest struct {
Page int `json:"page" query:"page" validate:"omitempty,min=1" minimum:"1" description:"页码"`
PageSize int `json:"page_size" query:"page_size" validate:"omitempty,min=1,max=100" minimum:"1" maximum:"100" description:"每页数量"`
Category string `json:"category" query:"category" validate:"omitempty,oneof=rate_limited_upstream terminal already_target transient business account_config" enum:"rate_limited_upstream,terminal,already_target,transient,business,account_config" description:"类别筛选 (rate_limited_upstream:上游频率限制, terminal:终态不可操作, already_target:已是目标状态, transient:临时故障, business:业务错误, account_config:账户配置错误)"`
Status *int `json:"status" query:"status" validate:"omitempty,oneof=0 1" enum:"0,1" description:"状态 (0:停用, 1:启用)"`
Keyword string `json:"keyword" query:"keyword" validate:"omitempty,max=100" maxLength:"100" description:"关键字模糊搜索"`
}
// SaveGatewayErrorRuleRequest 新建或修改上游错误分类规则请求
type SaveGatewayErrorRuleRequest struct {
ID uint `json:"-" description:"规则ID"`
Keyword string `json:"keyword" validate:"required,max=200" required:"true" maxLength:"200" description:"匹配关键字或正则"`
MatchType string `json:"match_type" validate:"required,oneof=contains prefix regex" required:"true" enum:"contains,prefix,regex" description:"匹配方式 (contains:包含, prefix:前缀, regex:正则)"`
Category string `json:"category" validate:"required,oneof=rate_limited_upstream terminal already_target transient business account_config" required:"true" enum:"rate_limited_upstream,terminal,already_target,transient,business,account_config" description:"类别 (rate_limited_upstream:上游频率限制, terminal:终态不可操作, already_target:已是目标状态, transient:临时故障, business:业务错误, account_config:账户配置错误)"`
Operations []GatewayOperation `json:"operations" validate:"omitempty,dive,oneof=query_flow query_status query_realname stop start" description:"适用操作,为空表示全部"`
Priority int `json:"priority" validate:"min=1,max=10000" minimum:"1" maximum:"10000" description:"匹配优先级,越小越先匹配"`
MaybeAccepted bool `json:"maybe_accepted" description:"上游可能已受理(如返回含调用成功、操作进行中、请求被合并)"`
AutoDisablePolling *bool `json:"auto_disable_polling" description:"命中后自动关闭该卡轮询(停复机不重试、回写网关扩展状态),仅类别为 terminal 时可勾选;更新时不传保留原值"`
Status *int `json:"status" validate:"omitempty,oneof=0 1" enum:"0,1" description:"状态 (0:停用, 1:启用),新建默认启用"`
Remark string `json:"remark" validate:"omitempty,max=500" maxLength:"500" description:"备注"`
}
// UpdateGatewayErrorRuleStatusRequest 启停上游错误分类规则请求
type UpdateGatewayErrorRuleStatusRequest struct {
ID uint `json:"-" path:"id" description:"规则ID"`
Status *int `json:"status" validate:"required,oneof=0 1" required:"true" enum:"0,1" description:"状态 (0:停用, 1:启用)"`
}
// GatewayErrorRuleResponse 上游错误分类规则响应
type GatewayErrorRuleResponse struct {
ID uint `json:"id" description:"规则ID"`
Keyword string `json:"keyword" description:"匹配关键字或正则"`
MatchType string `json:"match_type" description:"匹配方式 (contains:包含, prefix:前缀, regex:正则)"`
Category string `json:"category" description:"类别"`
CategoryName string `json:"category_name" description:"类别名称"`
Operations []string `json:"operations" description:"适用操作,为空表示全部"`
Priority int `json:"priority" description:"匹配优先级"`
MaybeAccepted bool `json:"maybe_accepted" description:"上游可能已受理"`
AutoDisablePolling bool `json:"auto_disable_polling" description:"命中后自动关闭该卡轮询"`
Status int `json:"status" description:"状态 (0:停用, 1:启用)"`
Remark string `json:"remark" description:"备注"`
CreatedAt time.Time `json:"created_at" description:"创建时间"`
UpdatedAt time.Time `json:"updated_at" description:"更新时间"`
}
// GatewayErrorRulePageResult 上游错误分类规则分页结果
type GatewayErrorRulePageResult struct {
List []GatewayErrorRuleResponse `json:"items" description:"规则列表"`
Total int64 `json:"total" description:"总数"`
Page int `json:"page" description:"当前页"`
PageSize int `json:"size" description:"每页数量"`
}
// TestGatewayErrorRuleRequest 测试上游错误分类请求
type TestGatewayErrorRuleRequest struct {
Operation string `json:"operation" validate:"required,oneof=query_flow query_status query_realname stop start" required:"true" enum:"query_flow,query_status,query_realname,stop,start" description:"操作 (query_flow:查流量, query_status:查状态, query_realname:查实名, stop:停机, start:复机)"`
Message string `json:"message" validate:"required,max=1000" required:"true" maxLength:"1000" description:"上游返回信息"`
Draft *SaveGatewayErrorRuleRequest `json:"draft" description:"未保存的规则草稿,传入时与已启用规则一起参与匹配(草稿按自身优先级排序)"`
}
// TestGatewayErrorRuleResponse 测试上游错误分类响应
type TestGatewayErrorRuleResponse struct {
Category string `json:"category" description:"命中类别"`
CategoryName string `json:"category_name" description:"命中类别名称"`
RuleID uint `json:"rule_id" description:"命中规则ID,未命中或命中草稿为 0"`
Keyword string `json:"keyword" description:"命中关键字,未命中为空"`
MaybeAccepted bool `json:"maybe_accepted" description:"上游可能已受理"`
AutoDisablePolling bool `json:"auto_disable_polling" description:"命中后自动关闭该卡轮询"`
Matched bool `json:"matched" description:"是否命中规则;未命中时按业务错误处理"`
}
// StopResumeRecordListRequest 停复机记录列表请求
type StopResumeRecordListRequest struct {
Page int `json:"page" query:"page" validate:"omitempty,min=1" minimum:"1" description:"页码"`
PageSize int `json:"page_size" query:"page_size" validate:"omitempty,min=1,max=100" minimum:"1" maximum:"100" description:"每页数量"`
ICCID string `json:"iccid" query:"iccid" validate:"omitempty,max=30" maxLength:"30" description:"ICCID"`
CarrierID *uint `json:"carrier_id" query:"carrier_id" validate:"omitempty" description:"运营商账户ID"`
Action string `json:"action" query:"action" validate:"omitempty,oneof=stop start" description:"动作 (stop:停机, start:复机)"`
Source string `json:"source" query:"source" validate:"omitempty,max=50" maxLength:"50" description:"来源"`
Result string `json:"result" query:"result" validate:"omitempty,oneof=success failed unknown already_target skipped" description:"结果 (success:成功, failed:失败, unknown:结果未知, already_target:已是目标状态, skipped:跳过)"`
ErrorCategory string `json:"error_category" query:"error_category" validate:"omitempty,max=30" maxLength:"30" description:"错误类别"`
StartTime time.Time `json:"start_time" query:"start_time" validate:"required" required:"true" description:"开始时间(必填,RFC3339)"`
EndTime time.Time `json:"end_time" query:"end_time" validate:"required" required:"true" description:"结束时间(必填,RFC3339,跨度不超过 31 天)"`
}
// CardStopResumeRecordListRequest 单卡停复机历史请求
type CardStopResumeRecordListRequest struct {
ID uint `json:"-" path:"id" description:"IoT卡ID"`
Page int `json:"page" query:"page" validate:"omitempty,min=1" minimum:"1" description:"页码"`
PageSize int `json:"page_size" query:"page_size" validate:"omitempty,min=1,max=100" minimum:"1" maximum:"100" description:"每页数量"`
StartTime time.Time `json:"start_time" query:"start_time" description:"开始时间(RFC3339),为空取最近 31 天"`
EndTime time.Time `json:"end_time" query:"end_time" description:"结束时间(RFC3339),为空取当前"`
}
// StopResumeRecordResponse 停复机记录响应
type StopResumeRecordResponse struct {
ID uint `json:"id" description:"记录ID"`
CardID uint `json:"card_id" description:"IoT卡ID"`
ICCID string `json:"iccid" description:"ICCID"`
CarrierID uint `json:"carrier_id" description:"运营商账户ID"`
Action string `json:"action" description:"动作 (stop:停机, start:复机)"`
ActionName string `json:"action_name" description:"动作名称"`
Source string `json:"source" description:"来源"`
Scene string `json:"scene" description:"场景"`
Reason string `json:"reason" description:"停机原因"`
CommandID string `json:"command_id" description:"停复机命令标识,同一命令的多次尝试相同"`
Attempt int `json:"attempt" description:"尝试序号"`
Result string `json:"result" description:"结果 (success:成功, failed:失败, unknown:结果未知, already_target:已是目标状态, skipped:跳过)"`
ErrorCategory string `json:"error_category" description:"错误类别"`
ErrorCategoryName string `json:"error_category_name" description:"错误类别名称"`
ProviderCode string `json:"provider_code" description:"上游返回码"`
ProviderMessage string `json:"provider_message" description:"上游返回信息(已截断)"`
DurationMS int `json:"duration_ms" description:"耗时(毫秒)"`
CreatedAt time.Time `json:"created_at" description:"记录时间"`
}
// StopResumeRecordPageResult 停复机记录分页结果
type StopResumeRecordPageResult struct {
List []StopResumeRecordResponse `json:"items" description:"记录列表"`
Total int64 `json:"total" description:"总数"`
Page int `json:"page" description:"当前页"`
PageSize int `json:"size" description:"每页数量"`
}
// GatewayCallStatsRequest 按运营商账户的网关调用统计请求
type GatewayCallStatsRequest struct {
Minutes int `json:"minutes" query:"minutes" validate:"omitempty,min=1,max=60" minimum:"1" maximum:"60" description:"统计最近多少分钟,默认 15,最大 60"`
}
// GatewayCallTrendRequest 单个运营商账户的网关调用趋势请求
type GatewayCallTrendRequest struct {
CarrierID uint `json:"-" path:"carrier_id" description:"运营商账户ID(0 表示无法解析账户的卡)"`
Minutes int `json:"minutes" query:"minutes" validate:"omitempty,min=1,max=10080" minimum:"1" maximum:"10080" description:"统计最近多少分钟,默认 60,最大 10080(7 天)"`
BucketMinutes int `json:"bucket_minutes" query:"bucket_minutes" validate:"omitempty,min=1,max=1440" minimum:"1" maximum:"1440" description:"聚合粒度(分钟),默认 1"`
}
// GatewayOperationStat 某操作在统计窗口内的调用汇总
type GatewayOperationStat struct {
Operation string `json:"operation" description:"操作 (query_flow:查流量, query_status:查状态, query_realname:查实名, stop:停机, start:复机)"`
OperationName string `json:"operation_name" description:"操作名称"`
Attempts int64 `json:"attempts" description:"实际发出的请求数"`
Success int64 `json:"success" description:"成功数"`
SuccessRate float64 `json:"success_rate" description:"成功率(0~1)"`
AvgQPS float64 `json:"avg_qps" description:"平均每秒请求数"`
PeakMinuteQPS float64 `json:"peak_minute_qps" description:"峰值分钟的平均每秒请求数"`
WouldLimit int64 `json:"would_limit" description:"超过限流组上限的次数(只统计模式下为本应被限流次数)"`
RateLimited int64 `json:"rate_limited" description:"被限流拒绝的次数(拦截模式)"`
Failures map[string]int64 `json:"failures" description:"按错误类别的失败数(network/timeout/http/invalid_response/upstream:<类别>)"`
Exempt map[string]int64 `json:"exempt" description:"按豁免来源的调用量 (manual_click/channel_threshold/auto_renewal/manual_refresh)"`
AvgDurationMS float64 `json:"avg_duration_ms" description:"平均耗时(毫秒)"`
P95DurationMS int64 `json:"p95_duration_ms" description:"p95 耗时上界(毫秒,按分桶近似),-1 表示超过最大分桶"`
}
// GatewayCarrierStat 某运营商账户的网关调用汇总
type GatewayCarrierStat struct {
CarrierID uint `json:"carrier_id" description:"运营商账户ID(0 表示无法解析账户的卡)"`
CarrierName string `json:"carrier_name" description:"运营商账户名称"`
Operations []GatewayOperationStat `json:"operations" description:"各操作汇总"`
}
// GatewayCallStatsResponse 按运营商账户的网关调用统计响应
type GatewayCallStatsResponse struct {
Mode string `json:"mode" description:"当前限流模式 (observe:只统计, enforce:拦截)"`
Minutes int `json:"minutes" description:"统计窗口(分钟)"`
Carriers []GatewayCarrierStat `json:"carriers" description:"有调用的运营商账户汇总"`
}
// GatewayCallTrendPoint 趋势中的一个时间桶
type GatewayCallTrendPoint struct {
Time string `json:"time" description:"时间桶起点(上海时区,yyyy-MM-dd HH:mm)"`
Operations []GatewayOperationStat `json:"operations" description:"各操作在该时间桶的汇总"`
}
// GatewayCallTrendResponse 单个运营商账户的网关调用趋势响应
type GatewayCallTrendResponse struct {
CarrierID uint `json:"carrier_id" description:"运营商账户ID"`
Points []GatewayCallTrendPoint `json:"points" description:"时间桶列表"`
}
// PollingOptionItem 枚举选项
type PollingOptionItem struct {
Value string `json:"value" description:"取值"`
Label string `json:"label" description:"中文名称"`
}
// PollingOptionsResponse 轮询与网关限流相关枚举
type PollingOptionsResponse struct {
GatewayOperations []PollingOptionItem `json:"gateway_operations" description:"卡网关操作"`
ErrorCategories []PollingOptionItem `json:"error_categories" description:"上游错误分类类别"`
MatchTypes []PollingOptionItem `json:"match_types" description:"错误分类匹配方式"`
RateLimitModes []PollingOptionItem `json:"rate_limit_modes" description:"限流模式"`
ExemptSources []PollingOptionItem `json:"exempt_sources" description:"限流豁免来源"`
StopResumeResults []PollingOptionItem `json:"stop_resume_results" description:"停复机记录结果"`
StopResumeActions []PollingOptionItem `json:"stop_resume_actions" description:"停复机动作"`
}
// UpdateRateLimitGroupParams 修改网关限流组参数(路径 + 请求体,供接口文档使用)
type UpdateRateLimitGroupParams struct {
IDReq
SaveRateLimitGroupRequest
}
// UpdateGatewayErrorRuleParams 修改上游错误分类规则参数(路径 + 请求体,供接口文档使用)
type UpdateGatewayErrorRuleParams struct {
IDReq
SaveGatewayErrorRuleRequest
}

View File

@@ -2,32 +2,39 @@ package dto
// CreatePollingConfigRequest 创建轮询配置请求
type CreatePollingConfigRequest struct {
ConfigName string `json:"config_name" validate:"required,min=1,max=100" required:"true" minLength:"1" maxLength:"100" description:"配置名称"`
CardCondition string `json:"card_condition" validate:"omitempty,oneof=not_real_name real_name activated suspended" description:"卡状态条件 (not_real_name:未实名, real_name:已实名, activated:已激活, suspended:已停用)"`
CardCategory string `json:"card_category" validate:"omitempty,oneof=normal industry" description:"卡业务类型 (normal:普通卡, industry:行业卡)"`
CarrierID *uint `json:"carrier_id" validate:"omitempty" description:"运营商ID(可选,精确匹配)"`
Priority int `json:"priority" validate:"required,min=1,max=1000" required:"true" minimum:"1" maximum:"1000" description:"优先级(数字越小优先级越高)"`
RealnameCheckInterval *int `json:"realname_check_interval" validate:"omitempty,min=30" minimum:"30" description:"实名检查间隔(秒),NULL表示不检查,最小30秒"`
CarddataCheckInterval *int `json:"carddata_check_interval" validate:"omitempty,min=60" minimum:"60" description:"流量检查间隔(秒),NULL表示不检查,最小60秒"`
PackageCheckInterval *int `json:"package_check_interval" validate:"omitempty,min=60" minimum:"60" description:"套餐检查间隔(秒),NULL表示不检查,最小60秒"`
ProtectCheckInterval *int `json:"protect_check_interval" validate:"omitempty,min=30" minimum:"30" description:"保护期一致性检查间隔(秒),NULL表示不检查,最小30秒"`
CardStatusCheckInterval *int `json:"card_status_check_interval" validate:"omitempty,min=30" minimum:"30" description:"卡状态检查间隔(秒),NULL表示不检查,最小30秒"`
Description string `json:"description" validate:"omitempty,max=500" maxLength:"500" description:"配置说明"`
ConfigName string `json:"config_name" validate:"required,min=1,max=100" required:"true" minLength:"1" maxLength:"100" description:"配置名称"`
CardCondition string `json:"card_condition" validate:"omitempty,oneof=not_real_name real_name activated suspended" description:"卡状态条件 (not_real_name:未实名, real_name:已实名, activated:已激活, suspended:已停用)"`
CardCategory string `json:"card_category" validate:"omitempty,oneof=normal industry" description:"卡业务类型 (normal:普通卡, industry:行业卡)"`
CarrierID *uint `json:"carrier_id" validate:"omitempty" description:"运营商ID(可选,精确匹配)"`
Priority int `json:"priority" validate:"required,min=1,max=1000" required:"true" minimum:"1" maximum:"1000" description:"优先级(数字越小优先级越高)"`
RealnameCheckInterval *int `json:"realname_check_interval" validate:"omitempty,min=0" minimum:"0" description:"实名检查间隔(秒):正数按间隔(最小30秒),0 不检查且不再继承,不传或为空继承优先级更低的配置"`
CarddataCheckInterval *int `json:"carddata_check_interval" validate:"omitempty,min=0" minimum:"0" description:"流量检查间隔(秒):正数按间隔(最小60秒),0 不检查且不再继承,不传或为空继承优先级更低的配置"`
PackageCheckInterval *int `json:"package_check_interval" validate:"omitempty,min=0" minimum:"0" description:"套餐检查间隔(秒):正数按间隔(最小60秒),0 不检查且不再继承,不传或为空继承优先级更低的配置"`
CardStatusCheckInterval *int `json:"card_status_check_interval" validate:"omitempty,min=0" minimum:"0" description:"卡状态检查间隔(秒):正数按间隔(最小30秒),0 不检查且不再继承,不传或为空继承优先级更低的配置"`
PackageState *string `json:"package_state" validate:"omitempty,oneof=active depleted pending none" enum:"active,depleted,pending,none" description:"套餐状态匹配 (active:有生效套餐, depleted:套餐已用完, pending:只有待生效套餐, none:无套餐),为空不限"`
AssetStatus *int `json:"asset_status" validate:"omitempty,oneof=1 2 3 4" enum:"1,2,3,4" description:"资产状态匹配 (1:在库, 2:已销售, 3:已换货, 4:已停用),为空不限"`
PreResetWindow *int `json:"pre_reset_window" validate:"omitempty,min=60" minimum:"60" description:"重置日前加密窗口长度(秒),与窗口内流量间隔同时配置才生效"`
PreResetCarddataInterval *int `json:"pre_reset_carddata_interval" validate:"omitempty,min=60" minimum:"60" description:"重置日前窗口内的流量检查间隔(秒),与窗口长度同时配置才生效"`
Description string `json:"description" validate:"omitempty,max=500" maxLength:"500" description:"配置说明"`
}
// UpdatePollingConfigRequest 更新轮询配置请求
type UpdatePollingConfigRequest struct {
ConfigName *string `json:"config_name" validate:"omitempty,min=1,max=100" minLength:"1" maxLength:"100" description:"配置名称"`
CardCondition *string `json:"card_condition" validate:"omitempty,oneof=not_real_name real_name activated suspended" description:"卡状态条件 (not_real_name:未实名, real_name:已实名, activated:已激活, suspended:已停用)"`
CardCategory *string `json:"card_category" validate:"omitempty,oneof=normal industry" description:"卡业务类型 (normal:普通卡, industry:行业卡)"`
CarrierID *uint `json:"carrier_id" validate:"omitempty" description:"运营商ID(可选,精确匹配)"`
Priority *int `json:"priority" validate:"omitempty,min=1,max=1000" minimum:"1" maximum:"1000" description:"优先级(数字越小优先级越高)"`
RealnameCheckInterval *int `json:"realname_check_interval" validate:"omitempty,min=30" minimum:"30" description:"实名检查间隔(秒),NULL表示不检查,最小30秒"`
CarddataCheckInterval *int `json:"carddata_check_interval" validate:"omitempty,min=60" minimum:"60" description:"流量检查间隔(秒),NULL表示不检查,最小60秒"`
PackageCheckInterval *int `json:"package_check_interval" validate:"omitempty,min=60" minimum:"60" description:"套餐检查间隔(秒),NULL表示不检查,最小60秒"`
ProtectCheckInterval *int `json:"protect_check_interval" validate:"omitempty,min=30" minimum:"30" description:"保护期一致性检查间隔(秒),NULL表示不检查,最小30秒"`
CardStatusCheckInterval *int `json:"card_status_check_interval" validate:"omitempty,min=30" minimum:"30" description:"卡状态检查间隔(秒),NULL表示不检查,最小30秒"`
Description *string `json:"description" validate:"omitempty,max=500" maxLength:"500" description:"配置说明"`
ConfigName *string `json:"config_name" validate:"omitempty,min=1,max=100" minLength:"1" maxLength:"100" description:"配置名称"`
CardCondition *string `json:"card_condition" validate:"omitempty,oneof=not_real_name real_name activated suspended" description:"卡状态条件 (not_real_name:未实名, real_name:已实名, activated:已激活, suspended:已停用)"`
CardCategory *string `json:"card_category" validate:"omitempty,oneof=normal industry" description:"卡业务类型 (normal:普通卡, industry:行业卡)"`
CarrierID *uint `json:"carrier_id" validate:"omitempty" description:"运营商ID(可选,精确匹配)"`
Priority *int `json:"priority" validate:"omitempty,min=1,max=1000" minimum:"1" maximum:"1000" description:"优先级(数字越小优先级越高)"`
RealnameCheckInterval *int `json:"realname_check_interval" validate:"omitempty,min=0" minimum:"0" description:"实名检查间隔(秒):正数按间隔(最小30秒),0 不检查且不再继承,不传或为空继承优先级更低的配置"`
CarddataCheckInterval *int `json:"carddata_check_interval" validate:"omitempty,min=0" minimum:"0" description:"流量检查间隔(秒):正数按间隔(最小60秒),0 不检查且不再继承,不传或为空继承优先级更低的配置"`
PackageCheckInterval *int `json:"package_check_interval" validate:"omitempty,min=0" minimum:"0" description:"套餐检查间隔(秒):正数按间隔(最小60秒),0 不检查且不再继承,不传或为空继承优先级更低的配置"`
CardStatusCheckInterval *int `json:"card_status_check_interval" validate:"omitempty,min=0" minimum:"0" description:"卡状态检查间隔(秒):正数按间隔(最小30秒),0 不检查且不再继承,不传或为空继承优先级更低的配置"`
PackageState *string `json:"package_state" validate:"omitempty,oneof=active depleted pending none" enum:"active,depleted,pending,none" description:"套餐状态匹配 (active:有生效套餐, depleted:套餐已用完, pending:只有待生效套餐, none:无套餐),为空不限"`
AssetStatus *int `json:"asset_status" validate:"omitempty,oneof=1 2 3 4" enum:"1,2,3,4" description:"资产状态匹配 (1:在库, 2:已销售, 3:已换货, 4:已停用),为空不限"`
PreResetWindow *int `json:"pre_reset_window" validate:"omitempty,min=60" minimum:"60" description:"重置日前加密窗口长度(秒),与窗口内流量间隔同时配置才生效"`
PreResetCarddataInterval *int `json:"pre_reset_carddata_interval" validate:"omitempty,min=60" minimum:"60" description:"重置日前窗口内的流量检查间隔(秒),与窗口长度同时配置才生效"`
ClearFields []PollingConfigClearField `json:"clear_fields" validate:"omitempty,dive,oneof=realname_check_interval carddata_check_interval package_check_interval card_status_check_interval carrier_id package_state asset_status pre_reset_window pre_reset_carddata_interval" description:"要清空为空值的字段(间隔清空即改为继承,匹配条件清空即不限):realname_check_interval/carddata_check_interval/package_check_interval/card_status_check_interval/carrier_id/package_state/asset_status/pre_reset_window/pre_reset_carddata_interval"`
Description *string `json:"description" validate:"omitempty,max=500" maxLength:"500" description:"配置说明"`
}
// PollingConfigListRequest 轮询配置列表请求
@@ -48,21 +55,25 @@ type UpdatePollingConfigStatusRequest struct {
// PollingConfigResponse 轮询配置响应
type PollingConfigResponse struct {
ID uint `json:"id" description:"配置ID"`
ConfigName string `json:"config_name" description:"配置名称"`
CardCondition string `json:"card_condition" description:"卡状态条件 (not_real_name:未实名, real_name:已实名, activated:已激活, suspended:已停用)"`
CardCategory string `json:"card_category" description:"卡业务类型 (normal:普通卡, industry:行业卡)"`
CarrierID *uint `json:"carrier_id" description:"运营商ID"`
Priority int `json:"priority" description:"优先级(数字越小优先级越高)"`
RealnameCheckInterval *int `json:"realname_check_interval" description:"实名检查间隔(秒),NULL表示不检查"`
CarddataCheckInterval *int `json:"carddata_check_interval" description:"流量检查间隔(秒),NULL表示不检查"`
PackageCheckInterval *int `json:"package_check_interval" description:"套餐检查间隔(秒),NULL表示不检查"`
ProtectCheckInterval *int `json:"protect_check_interval" description:"保护期一致性检查间隔(秒),NULL表示不检查"`
CardStatusCheckInterval *int `json:"card_status_check_interval" description:"卡状态检查间隔(秒),NULL表示不检查"`
Status int16 `json:"status" description:"状态 (1:启用, 0:禁用)"`
Description string `json:"description" description:"配置说明"`
CreatedAt string `json:"created_at" description:"创建时间"`
UpdatedAt string `json:"updated_at" description:"更新时间"`
ID uint `json:"id" description:"配置ID"`
ConfigName string `json:"config_name" description:"配置名称"`
CardCondition string `json:"card_condition" description:"卡状态条件 (not_real_name:未实名, real_name:已实名, activated:已激活, suspended:已停用)"`
CardCategory string `json:"card_category" description:"卡业务类型 (normal:普通卡, industry:行业卡)"`
CarrierID *uint `json:"carrier_id" description:"运营商ID"`
Priority int `json:"priority" description:"优先级(数字越小优先级越高)"`
RealnameCheckInterval *int `json:"realname_check_interval" description:"实名检查间隔(秒):正数按间隔,0 不检查,空继承"`
CarddataCheckInterval *int `json:"carddata_check_interval" description:"流量检查间隔(秒):正数按间隔,0 不检查,空继承"`
PackageCheckInterval *int `json:"package_check_interval" description:"套餐检查间隔(秒):正数按间隔,0 不检查,空继承"`
CardStatusCheckInterval *int `json:"card_status_check_interval" description:"卡状态检查间隔(秒):正数按间隔,0 不检查,空继承"`
PackageState *string `json:"package_state" description:"套餐状态匹配 (active:有生效套餐, depleted:套餐已用完, pending:只有待生效套餐, none:无套餐),为空不限"`
AssetStatus *int `json:"asset_status" description:"资产状态匹配 (1:在库, 2:已销售, 3:已换货, 4:已停用),为空不限"`
PreResetWindow *int `json:"pre_reset_window" description:"重置日前加密窗口长度(秒)"`
PreResetCarddataInterval *int `json:"pre_reset_carddata_interval" description:"重置日前窗口内的流量检查间隔(秒)"`
Warnings []string `json:"warnings,omitempty" description:"保存提示(如与其他启用配置优先级相同且匹配条件可能重叠),不影响保存结果"`
Status int16 `json:"status" description:"状态 (1:启用, 0:禁用)"`
Description string `json:"description" description:"配置说明"`
CreatedAt string `json:"created_at" description:"创建时间"`
UpdatedAt string `json:"updated_at" description:"更新时间"`
}
// UpdatePollingConfigParams 更新轮询配置参数
@@ -84,3 +95,150 @@ type PollingConfigPageResult struct {
Page int `json:"page" description:"当前页"`
PageSize int `json:"size" description:"每页数量"`
}
// UpdatePollingConfigPrioritiesRequest 整体调整轮询配置优先级请求
type UpdatePollingConfigPrioritiesRequest struct {
OrderedIDs []uint `json:"ordered_ids" validate:"required,min=1,max=1000,dive,min=1" required:"true" minItems:"1" maxItems:"1000" description:"按期望生效顺序排列的配置 ID 列表(排在前面的优先级更高);ID 须为正整数且不重复;系统按顺序把优先级重排为 1、2、3…,未列出的配置保持原优先级"`
}
// PollingConfigOverride 匹配预览中的未保存修改;ID 为 0 表示新增草稿,Disabled 为 true 表示假设该配置停用。
// 字段含义与创建轮询配置相同,均为可选;草稿名称为空时显示"未保存的配置"。
type PollingConfigOverride struct {
ID uint `json:"id" description:"要覆盖的配置 ID,0 表示新增一条草稿配置"`
Disabled bool `json:"disabled" description:"假设该配置停用(不参与匹配),为 true 时其余字段忽略"`
ConfigName string `json:"config_name" maxLength:"100" description:"草稿配置名称"`
CardCondition string `json:"card_condition" enum:"not_real_name,activated,suspended" description:"卡状态条件 (not_real_name:未实名, activated:已激活, suspended:已停用),为空不限"`
CardCategory string `json:"card_category" enum:"normal,industry" description:"卡业务类型 (normal:普通卡, industry:行业卡),为空不限"`
CarrierID *uint `json:"carrier_id" description:"运营商账户ID,为空不限"`
Priority int `json:"priority" minimum:"0" description:"优先级(数字越小越优先;与已有配置相同时排在已有配置之后)"`
RealnameCheckInterval *int `json:"realname_check_interval" minimum:"0" description:"实名检查间隔(秒),正数按间隔、0 不检查、空继承"`
CarddataCheckInterval *int `json:"carddata_check_interval" minimum:"0" description:"流量检查间隔(秒),正数按间隔、0 不检查、空继承"`
PackageCheckInterval *int `json:"package_check_interval" minimum:"0" description:"套餐检查间隔(秒),正数按间隔、0 不检查、空继承"`
CardStatusCheckInterval *int `json:"card_status_check_interval" minimum:"0" description:"卡状态检查间隔(秒),正数按间隔、0 不检查、空继承"`
PackageState *string `json:"package_state" enum:"active,depleted,pending,none" description:"套餐状态匹配 (active:有生效套餐, depleted:套餐已用完, pending:只有待生效套餐, none:无套餐),为空不限"`
AssetStatus *int `json:"asset_status" enum:"1,2,3,4" description:"资产状态匹配 (1:在库, 2:已销售, 3:已换货, 4:已停用),为空不限"`
PreResetWindow *int `json:"pre_reset_window" minimum:"60" description:"重置日前加密窗口长度(秒),与窗口内流量间隔同时配置才生效"`
PreResetCarddataInterval *int `json:"pre_reset_carddata_interval" minimum:"60" description:"重置日前窗口内的流量检查间隔(秒),与窗口长度同时配置才生效"`
}
// PollingConfigMatchPreviewQuery 按当前配置预览的查询参数
type PollingConfigMatchPreviewQuery struct {
ICCID string `json:"-" query:"iccid" validate:"required,max=30" required:"true" maxLength:"30" description:"要预览的卡 ICCID"`
}
// PollingConfigMatchPreviewRequest 带未保存修改的匹配预览请求(请求体)
type PollingConfigMatchPreviewRequest struct {
ICCID string `json:"iccid" validate:"required,max=30" required:"true" maxLength:"30" description:"要预览的卡 ICCID"`
Overrides []PollingConfigOverride `json:"overrides" maxItems:"20" description:"未保存的修改,不写任何数据"`
}
// PollingCardMatchFacts 卡的匹配维度取值
type PollingCardMatchFacts struct {
CardID uint `json:"card_id" description:"IoT卡ID"`
ICCID string `json:"iccid" description:"ICCID"`
CardCondition string `json:"card_condition" description:"卡状态条件 (not_real_name:未实名, activated:已激活, suspended:已停用)"`
CardCategory string `json:"card_category" description:"卡业务类型 (normal:普通卡, industry:行业卡)"`
CarrierID uint `json:"carrier_id" description:"运营商账户ID"`
AssetStatus int `json:"asset_status" description:"资产状态 (1:在库, 2:已销售, 3:已换货, 4:已停用)"`
PackageState string `json:"package_state" description:"套餐状态 (active:有生效套餐, depleted:套餐已用完, pending:只有待生效套餐, none:无套餐),解析失败为空"`
PackageStateName string `json:"package_state_name" description:"套餐状态名称"`
EnablePolling bool `json:"enable_polling" description:"是否开启轮询"`
}
// PollingConfigBrief 匹配到的配置摘要
type PollingConfigBrief struct {
ID uint `json:"id" description:"配置ID"`
ConfigName string `json:"config_name" description:"配置名称"`
Priority int `json:"priority" description:"优先级"`
Draft bool `json:"draft" description:"是否为预览中的未保存配置"`
}
// PollingIntervalPreview 某检查类型最终采用的间隔与来源
type PollingIntervalPreview struct {
TaskType string `json:"task_type" description:"检查类型 (polling:realname:实名, polling:carddata:流量, polling:package:套餐, polling:card_status:卡状态)"`
TaskTypeName string `json:"task_type_name" description:"检查类型名称"`
Mode string `json:"mode" description:"结果 (interval:按间隔检查, disabled:不检查, unmatched:没有任何配置给出取值,不检查)"`
Interval int `json:"interval" description:"间隔(秒),仅 mode=interval 时有值"`
ConfigID uint `json:"config_id" description:"给出取值的配置ID"`
ConfigName string `json:"config_name" description:"给出取值的配置名称"`
Inherited bool `json:"inherited" description:"是否继承自非首个命中的配置(首个命中配置该项为空)"`
}
// PollingConfigMatchPreviewResponse 匹配预览响应
type PollingConfigMatchPreviewResponse struct {
Card PollingCardMatchFacts `json:"card" description:"卡的匹配维度取值"`
Matched []PollingConfigBrief `json:"matched" description:"命中的配置(按优先级、ID 升序)"`
Intervals []PollingIntervalPreview `json:"intervals" description:"各检查类型最终结果"`
}
// PollingRecalculateDueRequest 立即重算到期时间请求
type PollingRecalculateDueRequest struct {
CarrierID *uint `json:"carrier_id" description:"只处理该运营商账户的卡,为空处理全部开启轮询的卡"`
}
// PollingRecalculateDueTaskIDRequest 重算任务进度查询路径参数
type PollingRecalculateDueTaskIDRequest struct {
TaskID string `json:"-" path:"task_id" required:"true" description:"重算任务ID"`
}
// PollingRecalculateDueResponse 重算到期时间任务进度
type PollingRecalculateDueResponse struct {
TaskID string `json:"task_id" description:"重算任务ID"`
Status string `json:"status" description:"状态 (pending:已入队, running:执行中, completed:已完成, failed:失败)"`
CarrierID string `json:"carrier_id" description:"限定的运营商账户ID,为空表示全部"`
Total int `json:"total" description:"需处理的卡数"`
Processed int `json:"processed" description:"已处理的卡数"`
Advanced int `json:"advanced" description:"被提前的检查项数"`
Error string `json:"error" description:"失败原因"`
CreatedAt string `json:"created_at" description:"创建时间"`
StartedAt string `json:"started_at" description:"开始时间"`
FinishedAt string `json:"finished_at" description:"结束时间"`
}
// PollingTierStatItem 按命中配置统计的卡数
type PollingTierStatItem struct {
ConfigID uint `json:"config_id" description:"首个命中的配置ID,0 表示未匹配任何配置"`
ConfigName string `json:"config_name" description:"首个命中的配置名称"`
Cards int `json:"cards" description:"卡数"`
}
// PollingTierStatsResponse 分层统计响应
type PollingTierStatsResponse struct {
TotalCards int `json:"total_cards" description:"开启轮询的卡数"`
ByConfig []PollingTierStatItem `json:"by_config" description:"按首个命中配置统计"`
ByPackageState map[string]int `json:"by_package_state" description:"按套餐状态统计 (active/depleted/pending/none/unknown)"`
UnmatchedInStock int `json:"unmatched_in_stock" description:"未匹配任何配置的在库卡数(应告警)"`
UnmatchedSoldNoPackage int `json:"unmatched_sold_no_package" description:"未匹配任何配置的已销售无套餐卡数(应告警)"`
ComputedAt string `json:"computed_at" description:"统计时间(结果缓存 5 分钟)"`
}
// PollingPreResetCycleStat 某运营商账户某个重置周期的加密效果
type PollingPreResetCycleStat struct {
ResetAt string `json:"reset_at" description:"重置时刻(上海时区)"`
ActivePackageCards int `json:"active_package_cards" description:"当前有生效套餐的轮询卡数"`
CheckedInWindow int64 `json:"checked_in_window" description:"窗口内查到过流量的卡数(近似去重计数)"`
NotCheckedInWindow int64 `json:"not_checked_in_window" description:"有生效套餐但窗口内未查到流量的卡数(近似,最小为 0)"`
LastOffsetBuckets map[string]int `json:"last_offset_buckets" description:"每张卡窗口内最后一次查询距重置时刻的分布(分钟):le_5/le_15/le_30/le_60/gt_60"`
}
// PollingPreResetCarrierStat 某运营商账户的重置日前加密效果
type PollingPreResetCarrierStat struct {
CarrierID uint `json:"carrier_id" description:"运营商账户ID"`
CarrierName string `json:"carrier_name" description:"运营商账户名称"`
Previous PollingPreResetCycleStat `json:"previous" description:"最近一次已过去的重置周期"`
Upcoming PollingPreResetCycleStat `json:"upcoming" description:"即将到来的重置周期(窗口未开始时计数为 0)"`
}
// PollingPreResetStatsResponse 重置日前加密效果统计响应
type PollingPreResetStatsResponse struct {
Carriers []PollingPreResetCarrierStat `json:"carriers" description:"有开启轮询卡的运营商账户"`
}
// PollingConfigClearField 是更新轮询配置时可清空为空值的字段名。
type PollingConfigClearField string
// Enum 返回可清空字段的全部取值,供接口文档生成数组元素枚举。
func (PollingConfigClearField) Enum() []interface{} {
return []interface{}{"realname_check_interval", "carddata_check_interval", "package_check_interval", "card_status_check_interval",
"carrier_id", "package_state", "asset_status", "pre_reset_window", "pre_reset_carddata_interval"}
}

View File

@@ -6,7 +6,7 @@ type CreateRefundRequest struct {
ActualReceivedAmount *int64 `json:"actual_received_amount" validate:"omitempty" description:"已废弃:实收金额由系统从原成功支付记录或订单实际收款派生并冻结,提交人填写无效"`
Method string `json:"method" validate:"required,oneof=original_route customer_account asset_wallet agent_wallet" required:"true" description:"退款方式 (original_route:原路退款, customer_account:客户收款信息退款, asset_wallet:退回资产钱包, agent_wallet:退回代理主钱包)"`
CustomerAccountInfo string `json:"customer_account_info" validate:"omitempty,max=1000" maxLength:"1000" description:"客户收款信息,仅客户收款信息退款方式必填,不得复用公司线下收款方式字典"`
RequestedRefundAmount int64 `json:"requested_refund_amount" validate:"required,min=1" required:"true" minimum:"1" description:"申请退款金额(分)"`
RequestedRefundAmount int64 `json:"requested_refund_amount" validate:"min=0" required:"true" minimum:"0" description:"申请退款金额(分),0 ~ 冻结实收金额;0 元退款表示作废权益不退钱,仅限客户收款信息方式(钱包支付订单不支持),此时客户收款信息可不填但凭证必传"`
RefundVoucherKey []string `json:"refund_voucher_key" validate:"omitempty,max=5,dive,max=500" maxItems:"5" description:"退款凭证对象存储file_key列表(最多5个,仅客户收款信息退款方式必填,通过/storage/upload-url上传图片后获得)"`
RefundReason string `json:"refund_reason" validate:"required,max=1000" required:"true" maxLength:"1000" description:"退款原因(必填,去除首尾空白后不得为空)"`
// Remark 是申请人备注,随本次提交冻结进当次审批尝试快照;与审批备注(详情响应中的 remark)语义不同。
@@ -32,7 +32,7 @@ type ResubmitRefundRequest struct {
ActualReceivedAmount *int64 `json:"actual_received_amount" validate:"omitempty,min=1" minimum:"1" description:"已废弃:实收金额由系统从原成功支付记录或订单实际收款派生并冻结,提交人填写无效"`
Method *string `json:"method" validate:"omitempty,oneof=original_route customer_account asset_wallet agent_wallet" description:"退款方式 (original_route:原路退款, customer_account:客户收款信息退款, asset_wallet:退回资产钱包, agent_wallet:退回代理主钱包),不填沿用原有方式"`
CustomerAccountInfo *string `json:"customer_account_info" validate:"omitempty,max=1000" maxLength:"1000" description:"客户收款信息,仅客户收款信息退款方式必填,不得复用公司线下收款方式字典"`
RequestedRefundAmount *int64 `json:"requested_refund_amount" validate:"omitempty,min=1" minimum:"1" description:"申请退款金额(分)"`
RequestedRefundAmount *int64 `json:"requested_refund_amount" validate:"omitempty,min=0" minimum:"0" description:"申请退款金额(分),不填沿用原金额;0 元退款规则同创建:仅限客户收款信息方式(钱包支付订单不支持),客户收款信息可不填但凭证必传"`
RefundVoucherKey *[]string `json:"refund_voucher_key" validate:"omitempty,max=5,dive,max=500" maxItems:"5" description:"退款凭证对象存储file_key列表(重新提交时可替换;客户收款信息退款方式必填,最多5个)"`
RefundReason *string `json:"refund_reason" validate:"omitempty,max=1000" maxLength:"1000" description:"退款原因(不填沿用原有原因;重提时去除首尾空白后不得为空)"`
// Remark 是本次重提的申请人备注,随本次尝试冻结;不填沿用该退款单最近一次尝试的备注。

View File

@@ -0,0 +1,70 @@
package dto
// StopResumeRuleDryRunRequest 停复机规则演练请求。
type StopResumeRuleDryRunRequest struct {
CarrierID *uint `json:"carrier_id" query:"carrier_id" validate:"omitempty,min=1" minimum:"1" description:"运营商账户ID,不传为全部"`
StopReason string `json:"stop_reason" query:"stop_reason" validate:"omitempty,max=50" maxLength:"50" description:"停机原因筛选:传停机原因编码(如 not_realname、carrier_stopped、manual),传 none 表示停机原因为空,不传为全部"`
AssumeControlOn string `json:"assume_control_on" query:"assume_control_on" validate:"omitempty,max=2000" maxLength:"2000" description:"假设开启实名管控的运营商账户ID,逗号分隔;只用于预演,不改实际配置"`
AssumeControlOff string `json:"assume_control_off" query:"assume_control_off" validate:"omitempty,max=2000" maxLength:"2000" description:"假设关闭实名管控的运营商账户ID,逗号分隔;只用于预演,不改实际配置"`
Result string `json:"result" query:"result" validate:"omitempty,oneof=will_resume will_stop no_action" enum:"will_resume,will_stop,no_action" description:"明细按判断结果筛选 (will_resume:将复机, will_stop:将停机, no_action:不动作),汇总不受影响"`
Page int `json:"page" query:"page" validate:"omitempty,min=1" minimum:"1" description:"明细页码,默认1"`
PageSize int `json:"page_size" query:"page_size" validate:"omitempty,min=1,max=500" minimum:"1" maximum:"500" description:"明细每页数量,默认20,最大500"`
}
// StopResumeRuleDryRunExportRequest 停复机规则演练明细导出请求。
type StopResumeRuleDryRunExportRequest struct {
CarrierID *uint `json:"carrier_id" validate:"omitempty,min=1" minimum:"1" description:"运营商账户ID,不传为全部"`
StopReason string `json:"stop_reason" validate:"omitempty,max=50" maxLength:"50" description:"停机原因筛选,none 表示停机原因为空"`
AssumeControlOn string `json:"assume_control_on" validate:"omitempty,max=2000" maxLength:"2000" description:"假设开启实名管控的运营商账户ID,逗号分隔"`
AssumeControlOff string `json:"assume_control_off" validate:"omitempty,max=2000" maxLength:"2000" description:"假设关闭实名管控的运营商账户ID,逗号分隔"`
Format string `json:"format" validate:"omitempty,oneof=xlsx csv" enum:"xlsx,csv" description:"导出格式 (xlsx:Excel, csv:CSV),默认 xlsx"`
}
// StopResumeRuleDryRunSummaryItem 演练汇总项:运营商账户 × 当前停机原因 × 判断结果 × 原因。
type StopResumeRuleDryRunSummaryItem struct {
CarrierID uint `json:"carrier_id" description:"运营商账户ID"`
CarrierName string `json:"carrier_name" description:"运营商账户名称"`
StopReason string `json:"stop_reason" description:"当前停机原因,空表示无"`
StopReasonName string `json:"stop_reason_name" description:"当前停机原因名称"`
Result string `json:"result" description:"判断结果 (will_resume:将复机, will_stop:将停机, no_action:不动作)"`
ResultName string `json:"result_name" description:"判断结果名称"`
Reason string `json:"reason" description:"将停机时为目标停机原因,不动作时为不动作原因 (polling_suspended:暂停轮询, conditions_met:满足开机条件, no_package:无套餐, traffic_exhausted:流量已用完, not_realname:未实名, threshold_locked:持通道阈值锁, carrier_missing:所属通道不存在)"`
ReasonName string `json:"reason_name" description:"原因名称"`
Count int64 `json:"count" description:"卡数"`
}
// StopResumeRuleDryRunTotals 演练总计。
type StopResumeRuleDryRunTotals struct {
Total int64 `json:"total" description:"参与演练的开机与停机卡数"`
WillResume int64 `json:"will_resume" description:"将复机卡数"`
WillStop int64 `json:"will_stop" description:"将停机卡数"`
NoAction int64 `json:"no_action" description:"不动作卡数"`
}
// StopResumeRuleDryRunItem 演练明细。
type StopResumeRuleDryRunItem struct {
CardID uint `json:"card_id" description:"卡ID"`
ICCID string `json:"iccid" description:"ICCID"`
CarrierID uint `json:"carrier_id" description:"运营商账户ID"`
CarrierName string `json:"carrier_name" description:"运营商账户名称"`
DeviceID uint `json:"device_id" description:"有效绑定设备ID,独立卡为 0"`
NetworkStatus int `json:"network_status" description:"当前网络状态 (0:停机, 1:开机)"`
StopReason string `json:"stop_reason" description:"当前停机原因"`
StopReasonName string `json:"stop_reason_name" description:"当前停机原因名称"`
CardCategory string `json:"card_category" description:"卡类别"`
RealnameOK bool `json:"realname_ok" description:"按(假设的)通道实名管控是否满足实名条件"`
Result string `json:"result" description:"判断结果"`
ResultName string `json:"result_name" description:"判断结果名称"`
Reason string `json:"reason" description:"将停机时为目标停机原因,不动作时为不动作原因"`
ReasonName string `json:"reason_name" description:"原因名称"`
}
// StopResumeRuleDryRunResponse 停复机规则演练响应。
type StopResumeRuleDryRunResponse struct {
Totals StopResumeRuleDryRunTotals `json:"totals" description:"总计"`
Summary []StopResumeRuleDryRunSummaryItem `json:"summary" description:"按运营商账户 × 当前停机原因 × 判断结果 × 原因汇总"`
Items []StopResumeRuleDryRunItem `json:"items" description:"明细(按卡ID升序分页)"`
Total int64 `json:"total" description:"明细总数(按 result 筛选后)"`
Page int `json:"page" description:"当前页"`
PageSize int `json:"size" description:"每页数量"`
}

View File

@@ -4,7 +4,7 @@ import "time"
// SystemConfigListRequest 是系统配置分页查询参数。
type SystemConfigListRequest struct {
Module string `json:"module" query:"module" validate:"omitempty,oneof=carrier_callback c2b.payment" enum:"carrier_callback,c2b.payment" description:"模块筛选 (carrier_callback:运营商回调配置, c2b.payment:C端支付方式配置)"`
Module string `json:"module" query:"module" validate:"omitempty,oneof=carrier_callback c2b.payment polling" enum:"carrier_callback,c2b.payment,polling" description:"模块筛选 (carrier_callback:运营商回调配置, c2b.payment:C端支付方式配置, polling:轮询与网关限流配置)"`
Page int `json:"page" query:"page" validate:"omitempty,min=1" minimum:"1" description:"页码"`
PageSize int `json:"page_size" query:"page_size" validate:"omitempty,min=1,max=100" minimum:"1" maximum:"100" description:"每页数量"`
}
@@ -14,7 +14,7 @@ type SystemConfigItem struct {
ConfigKey string `json:"config_key" description:"稳定配置 Key"`
Value string `json:"value" description:"配置值;敏感值按注册策略脱敏"`
ValueType string `json:"value_type" description:"值类型 (string:字符串, int:整数, bool:布尔, json:JSON)"`
Module string `json:"module" enum:"carrier_callback,c2b.payment" description:"所属模块 (carrier_callback:运营商回调配置, c2b.payment:C端支付方式配置)"`
Module string `json:"module" enum:"carrier_callback,c2b.payment,polling" description:"所属模块 (carrier_callback:运营商回调配置, c2b.payment:C端支付方式配置, polling:轮询与网关限流配置)"`
Description string `json:"description" description:"中文说明"`
Readonly bool `json:"readonly" description:"是否只读"`
Sensitive bool `json:"sensitive" description:"是否敏感"`

View File

@@ -0,0 +1,74 @@
package model
import (
"time"
"gorm.io/datatypes"
"gorm.io/gorm"
)
// CarrierRateLimitGroup 运营商账户网关限流组。
// CarrierID 为 nil 表示全局组;Operations、HighPriorityOperations 为卡网关操作字符串数组(JSON)。
type CarrierRateLimitGroup struct {
gorm.Model
BaseModel `gorm:"embedded"`
CarrierID *uint `gorm:"column:carrier_id;comment:运营商账户ID,为空表示全局组" json:"carrier_id"`
GroupCode string `gorm:"column:group_code;type:varchar(50);not null;comment:组标识" json:"group_code"`
GroupName string `gorm:"column:group_name;type:varchar(100);not null;comment:组名称" json:"group_name"`
MaxQPS int `gorm:"column:max_qps;not null;comment:每秒请求上限" json:"max_qps"`
Operations datatypes.JSON `gorm:"column:operations;type:jsonb;not null;comment:组内操作" json:"operations"`
HighPriorityOperations datatypes.JSON `gorm:"column:high_priority_operations;type:jsonb;not null;comment:高优先级操作" json:"high_priority_operations"`
ReservedQPS int `gorm:"column:reserved_qps;not null;default:0;comment:预留给高优先级操作的额度" json:"reserved_qps"`
Status int `gorm:"column:status;type:smallint;not null;default:1;comment:状态 0-停用 1-启用" json:"status"`
}
// TableName 返回限流组表名。
func (CarrierRateLimitGroup) TableName() string {
return "tb_carrier_rate_limit_group"
}
// GatewayErrorClassification 上游错误分类规则。
type GatewayErrorClassification struct {
gorm.Model
BaseModel `gorm:"embedded"`
Keyword string `gorm:"column:keyword;type:varchar(200);not null;comment:匹配关键字或正则" json:"keyword"`
MatchType string `gorm:"column:match_type;type:varchar(20);not null;default:contains;comment:匹配方式" json:"match_type"`
Category string `gorm:"column:category;type:varchar(30);not null;comment:类别" json:"category"`
Operations datatypes.JSON `gorm:"column:operations;type:jsonb;not null;comment:适用操作,空数组表示全部" json:"operations"`
Priority int `gorm:"column:priority;not null;default:100;comment:匹配优先级" json:"priority"`
MaybeAccepted bool `gorm:"column:maybe_accepted;not null;default:false;comment:上游可能已受理" json:"maybe_accepted"`
// AutoDisablePolling 命中后自动关闭该卡轮询,仅终态不可操作类别可置位。
AutoDisablePolling bool `gorm:"column:auto_disable_polling;not null;default:false;comment:命中后自动关闭该卡轮询" json:"auto_disable_polling"`
Status int `gorm:"column:status;type:smallint;not null;default:1;comment:状态 0-停用 1-启用" json:"status"`
Remark string `gorm:"column:remark;type:varchar(500);not null;default:'';comment:备注" json:"remark"`
}
// TableName 返回错误分类规则表名。
func (GatewayErrorClassification) TableName() string {
return "tb_gateway_error_classification"
}
// CardStopResumeRecord 停复机调用记录(按月分区表),每次实际调用上游停机或复机一条。
type CardStopResumeRecord struct {
ID uint `gorm:"column:id;primaryKey;autoIncrement" json:"id"`
CardID uint `gorm:"column:card_id;not null" json:"card_id"`
ICCID string `gorm:"column:iccid;type:varchar(30);not null;default:''" json:"iccid"`
CarrierID uint `gorm:"column:carrier_id;not null;default:0" json:"carrier_id"`
Action string `gorm:"column:action;type:varchar(10);not null" json:"action"`
Source string `gorm:"column:source;type:varchar(50);not null;default:''" json:"source"`
Scene string `gorm:"column:scene;type:varchar(50);not null;default:''" json:"scene"`
Reason string `gorm:"column:reason;type:varchar(50);not null;default:''" json:"reason"`
CommandID string `gorm:"column:command_id;type:varchar(64);not null;default:''" json:"command_id"`
Attempt int `gorm:"column:attempt;not null;default:1" json:"attempt"`
Result string `gorm:"column:result;type:varchar(20);not null" json:"result"`
ErrorCategory string `gorm:"column:error_category;type:varchar(30);not null;default:''" json:"error_category"`
ProviderCode string `gorm:"column:provider_code;type:varchar(30);not null;default:''" json:"provider_code"`
ProviderMessage string `gorm:"column:provider_message;type:varchar(500);not null;default:''" json:"provider_message"`
DurationMS int `gorm:"column:duration_ms;not null;default:0" json:"duration_ms"`
CreatedAt time.Time `gorm:"column:created_at;type:timestamptz;primaryKey;not null" json:"created_at"`
}
// TableName 返回停复机记录表名。
func (CardStopResumeRecord) TableName() string {
return "tb_card_stop_resume_record"
}

View File

@@ -38,7 +38,6 @@ type IotCard struct {
LastRealNameCheckAt *time.Time `gorm:"column:last_real_name_check_at;comment:最后一次实名检查时间" json:"last_real_name_check_at"`
RealnameReversalCount int `gorm:"column:realname_reversal_count;type:int;default:0;not null;comment:实名逆转连续观测次数" json:"realname_reversal_count"`
RealnameReversalStartedAt *time.Time `gorm:"column:realname_reversal_started_at;comment:实名逆转当前确认窗口开始时间" json:"realname_reversal_started_at,omitempty"`
LastProtectCheckAt *time.Time `gorm:"column:last_protect_check_at;comment:上次保护期一致性检查时间" json:"last_protect_check_at"`
LastCardStatusCheckAt *time.Time `gorm:"column:last_card_status_check_at;comment:最后一次卡状态检查时间" json:"last_card_status_check_at"`
LastSyncTime *time.Time `gorm:"column:last_sync_time;comment:最后一次与Gateway同步时间" json:"last_sync_time"`
SeriesID *uint `gorm:"column:series_id;index;comment:套餐系列ID(关联PackageSeries)" json:"series_id,omitempty"`

View File

@@ -8,23 +8,26 @@ import (
// PollingConfig 轮询配置表
type PollingConfig struct {
ID uint `gorm:"column:id;primaryKey;autoIncrement" json:"id"`
ConfigName string `gorm:"column:config_name;type:varchar(100);not null;comment:配置名称" json:"config_name"`
CardCondition string `gorm:"column:card_condition;type:varchar(50);comment:卡状态条件:not_real_name/real_name/activated/suspended" json:"card_condition"`
CardCategory string `gorm:"column:card_category;type:varchar(50);comment:卡业务类型:normal/industry" json:"card_category"`
CarrierID *uint `gorm:"column:carrier_id;comment:运营商ID(可选,精确匹配)" json:"carrier_id"`
Priority int `gorm:"column:priority;not null;default:100;comment:优先级(数字越小优先级越高)" json:"priority"`
RealnameCheckInterval *int `gorm:"column:realname_check_interval;comment:实名检查间隔(秒),NULL表示不检查" json:"realname_check_interval"`
CarddataCheckInterval *int `gorm:"column:carddata_check_interval;comment:流量检查间隔(秒),NULL表示不检查" json:"carddata_check_interval"`
PackageCheckInterval *int `gorm:"column:package_check_interval;comment:套餐检查间隔(秒),NULL表示不检查" json:"package_check_interval"`
ProtectCheckInterval *int `gorm:"column:protect_check_interval;comment:保护期一致性检查间隔(秒),NULL=不参与" json:"protect_check_interval"`
CardStatusCheckInterval *int `gorm:"column:card_status_check_interval;comment:卡状态检查间隔(秒),NULL表示不启用卡状态轮询" json:"card_status_check_interval"`
Status int16 `gorm:"column:status;type:smallint;not null;default:1;comment:状态:0-禁用,1-启用" json:"status"`
Description string `gorm:"column:description;type:text;comment:配置说明" json:"description"`
CreatedAt time.Time `gorm:"column:created_at;not null;default:CURRENT_TIMESTAMP;comment:创建时间" json:"created_at"`
UpdatedAt time.Time `gorm:"column:updated_at;not null;default:CURRENT_TIMESTAMP;comment:更新时间" json:"updated_at"`
CreatedBy *uint `gorm:"column:created_by;comment:创建人ID" json:"created_by"`
UpdatedBy *uint `gorm:"column:updated_by;comment:更新人ID" json:"updated_by"`
ID uint `gorm:"column:id;primaryKey;autoIncrement" json:"id"`
ConfigName string `gorm:"column:config_name;type:varchar(100);not null;comment:配置名称" json:"config_name"`
CardCondition string `gorm:"column:card_condition;type:varchar(50);comment:卡状态条件:not_real_name/real_name/activated/suspended" json:"card_condition"`
CardCategory string `gorm:"column:card_category;type:varchar(50);comment:卡业务类型:normal/industry" json:"card_category"`
CarrierID *uint `gorm:"column:carrier_id;comment:运营商ID(可选,精确匹配)" json:"carrier_id"`
Priority int `gorm:"column:priority;not null;default:100;comment:优先级(数字越小优先级越高)" json:"priority"`
RealnameCheckInterval *int `gorm:"column:realname_check_interval;comment:实名检查间隔(秒):正数按间隔,0不检查(停止继承),NULL继承" json:"realname_check_interval"`
CarddataCheckInterval *int `gorm:"column:carddata_check_interval;comment:流量检查间隔(秒):正数按间隔,0不检查(停止继承),NULL继承" json:"carddata_check_interval"`
PackageCheckInterval *int `gorm:"column:package_check_interval;comment:套餐检查间隔(秒):正数按间隔,0不检查(停止继承),NULL继承" json:"package_check_interval"`
CardStatusCheckInterval *int `gorm:"column:card_status_check_interval;comment:卡状态检查间隔(秒):正数按间隔,0不检查(停止继承),NULL继承" json:"card_status_check_interval"`
PackageState *string `gorm:"column:package_state;type:varchar(20);comment:套餐状态匹配:active/depleted/pending/none,NULL不限" json:"package_state"`
AssetStatus *int `gorm:"column:asset_status;type:smallint;comment:资产状态匹配:1在库 2已销售 3已换货 4已停用,NULL不限" json:"asset_status"`
PreResetWindow *int `gorm:"column:pre_reset_window;comment:重置日前加密窗口长度(秒),NULL不加密" json:"pre_reset_window"`
PreResetCarddataInterval *int `gorm:"column:pre_reset_carddata_interval;comment:重置日前窗口内流量检查间隔(秒),NULL不加密" json:"pre_reset_carddata_interval"`
Status int16 `gorm:"column:status;type:smallint;not null;default:1;comment:状态:0-禁用,1-启用" json:"status"`
Description string `gorm:"column:description;type:text;comment:配置说明" json:"description"`
CreatedAt time.Time `gorm:"column:created_at;not null;default:CURRENT_TIMESTAMP;comment:创建时间" json:"created_at"`
UpdatedAt time.Time `gorm:"column:updated_at;not null;default:CURRENT_TIMESTAMP;comment:更新时间" json:"updated_at"`
CreatedBy *uint `gorm:"column:created_by;comment:创建人ID" json:"created_by"`
UpdatedBy *uint `gorm:"column:updated_by;comment:更新人ID" json:"updated_by"`
}
// TableName 指定表名

View File

@@ -0,0 +1,226 @@
package polling
import (
"context"
"strconv"
"sync"
"sync/atomic"
"time"
"github.com/redis/go-redis/v9"
"go.uber.org/zap"
"github.com/break/junhong_cmp_fiber/internal/model"
"github.com/break/junhong_cmp_fiber/internal/polling/carrierqueue"
"github.com/break/junhong_cmp_fiber/internal/store/postgres"
"github.com/break/junhong_cmp_fiber/pkg/constants"
)
// CarrierScheduling 是按运营商账户调度在轮询包内的接缝:判断当前模式,并按卡计算各检查类型到期时间写入账户队列。
// 所有写入轮询结构的入口(初始化、生命周期、优先提示、手动触发)按当前模式写新结构或旧分片结构。
type CarrierScheduling struct {
store *carrierqueue.Store
settings *carrierqueue.SettingsProvider
configMgr *PollingConfigManager
logger *zap.Logger
}
// NewCarrierScheduling 创建按运营商账户调度接缝。
func NewCarrierScheduling(store *carrierqueue.Store, settings *carrierqueue.SettingsProvider, configMgr *PollingConfigManager, logger *zap.Logger) *CarrierScheduling {
if logger == nil {
logger = zap.NewNop()
}
return &CarrierScheduling{store: store, settings: settings, configMgr: configMgr, logger: logger}
}
// Active 判断当前是否为按运营商账户调度。
func (c *CarrierScheduling) Active(ctx context.Context) bool {
return c != nil && c.store != nil && c.settings != nil && c.settings.Current(ctx).CarrierMode()
}
// Store 返回账户队列存储。
func (c *CarrierScheduling) Store() *carrierqueue.Store {
if c == nil {
return nil
}
return c.store
}
// Settings 返回调度参数提供者。
func (c *CarrierScheduling) Settings() *carrierqueue.SettingsProvider {
if c == nil {
return nil
}
return c.settings
}
// DuesForCard 按卡匹配的轮询配置计算各检查类型下次到期时间。
// fromLastCheck 为 true 时按各类型上次检查时间计算(初始化);否则按当前时间加抖动(新卡、恢复轮询、状态变化)。
// 间隔为 0 或未配置的类型不返回;重置日前加密窗口规则算出的时间更早时以它为准。
func (c *CarrierScheduling) DuesForCard(ctx context.Context, card *model.IotCard, fromLastCheck bool) map[string]time.Time {
dues := map[string]time.Time{}
if c == nil || c.configMgr == nil || card == nil {
return dues
}
now := time.Now()
for taskType, info := range c.configMgr.MergedTaskIntervals(card) {
checkType := constants.CheckTypeFromTaskType(taskType)
if checkType == "" || info.Interval <= 0 {
continue
}
var due time.Time
if fromLastCheck {
due = calculateNextCheckTime(lastCheckAtByTaskType(card, taskType), info.Interval, now)
if windowDue := c.configMgr.NextCheckAt(ctx, card, taskType, info, now); windowDue.Before(due) && windowDue.After(now) {
due = windowDue
}
} else {
due = calcInitialDelay(info.Interval)
}
dues[checkType] = due
}
return dues
}
// UpsertCard 把卡写入所属账户队列;fromLastCheck 含义同 DuesForCard。
func (c *CarrierScheduling) UpsertCard(ctx context.Context, card *model.IotCard, fromLastCheck bool) {
if c == nil || c.store == nil || card == nil {
return
}
if err := c.store.Upsert(ctx, card.ID, card.CarrierID, c.DuesForCard(ctx, card, fromLastCheck)); err != nil {
c.logger.Warn("写入运营商账户轮询队列失败", zap.Uint("card_id", card.ID), zap.Uint("carrier_id", card.CarrierID), zap.Error(err))
}
}
// RemoveCard 把卡移出所属账户队列;carrierID 为 0 时按到期信息中记录的账户移出。
func (c *CarrierScheduling) RemoveCard(ctx context.Context, cardID, carrierID uint) {
if c == nil || c.store == nil {
return
}
if carrierID == 0 {
due, err := c.store.ReadDue(ctx, cardID)
if err != nil || due.CarrierID == 0 {
return
}
carrierID = due.CarrierID
}
if err := c.store.Remove(ctx, cardID, carrierID); err != nil {
c.logger.Warn("移出运营商账户轮询队列失败", zap.Uint("card_id", cardID), zap.Error(err))
}
}
// RequestTaskType 把卡的某检查类型(旧任务类型 polling:xxx)设为立即到期放入账户优先队列(优先提示、手动触发)。
// 卡不在轮询中时不处理。
func (c *CarrierScheduling) RequestTaskType(ctx context.Context, cardID, carrierID uint, taskType string) {
checkType := constants.CheckTypeFromTaskType(taskType)
if c == nil || c.store == nil || checkType == "" {
return
}
if _, err := c.store.RequestCheck(ctx, carrierqueue.CheckRequest{CardID: cardID, CarrierID: carrierID, CheckTypes: []string{checkType}}); err != nil {
c.logger.Warn("写入运营商账户优先队列失败", zap.Uint("card_id", cardID), zap.String("task_type", taskType), zap.Error(err))
}
}
// CarrierInitializer 从数据库按批初始化全部运营商账户队列与卡到期信息;进度与旧初始化器同结构,供 init-progress 展示。
type CarrierInitializer struct {
scheduling *CarrierScheduling
iotCardStore *postgres.IotCardStore
logger *zap.Logger
mu sync.RWMutex
progress InitProgress
running atomic.Bool
}
// NewCarrierInitializer 创建新结构初始化器。
func NewCarrierInitializer(scheduling *CarrierScheduling, iotCardStore *postgres.IotCardStore, logger *zap.Logger) *CarrierInitializer {
if logger == nil {
logger = zap.NewNop()
}
return &CarrierInitializer{scheduling: scheduling, iotCardStore: iotCardStore, logger: logger}
}
// Progress 返回初始化进度快照。
func (i *CarrierInitializer) Progress() InitProgress {
i.mu.RLock()
defer i.mu.RUnlock()
return i.progress
}
// Run 同步执行一次全量初始化:按批读取启用轮询的卡,计算各类型到期并写入账户队列。返回写入的卡数。
func (i *CarrierInitializer) Run(ctx context.Context) (int64, error) {
if !i.running.CompareAndSwap(false, true) {
return 0, nil
}
defer i.running.Store(false)
started := time.Now()
total, err := i.iotCardStore.CountForPolling(ctx)
if err != nil {
i.setProgress(func(p *InitProgress) { p.Status, p.ErrorMessage = "failed", err.Error() })
return 0, err
}
i.setProgress(func(p *InitProgress) {
*p = InitProgress{TotalCards: total, StartTime: started, Status: "running"}
})
const batchSize = 2000
var lastID uint
var written int64
for {
if err := ctx.Err(); err != nil {
i.setProgress(func(p *InitProgress) { p.Status, p.ErrorMessage = "failed", err.Error() })
return written, err
}
cards, fetchErr := i.iotCardStore.ListForPollingBatch(ctx, lastID, batchSize)
if fetchErr != nil {
i.setProgress(func(p *InitProgress) { p.Status, p.ErrorMessage = "failed", fetchErr.Error() })
return written, fetchErr
}
if len(cards) == 0 {
break
}
if tier := i.scheduling.configMgr.TierResolver(); tier != nil {
tier.Prefill(ctx, cards)
}
for _, card := range cards {
i.scheduling.UpsertCard(ctx, card, true)
written++
}
lastID = cards[len(cards)-1].ID
loaded := written
i.setProgress(func(p *InitProgress) { p.LoadedCards, p.LastBatchTime = loaded, time.Now() })
}
i.setProgress(func(p *InitProgress) { p.Status = "completed" })
i.logger.Info("运营商账户轮询结构初始化完成", zap.Int64("cards", written), zap.Duration("duration", time.Since(started)))
return written, nil
}
func (i *CarrierInitializer) setProgress(update func(*InitProgress)) {
i.mu.Lock()
update(&i.progress)
i.mu.Unlock()
}
// AdvanceDue 把卡某检查类型的到期时间提前到 due(只提前不推后),并同步把账户队列中的 score 提前;返回是否提前。
// 卡不在轮询中(无到期信息)时不处理。
func (c *CarrierScheduling) AdvanceDue(ctx context.Context, card *model.IotCard, checkType string, due time.Time) (bool, error) {
if c == nil || c.store == nil || card == nil {
return false, nil
}
current, err := c.store.ReadDue(ctx, card.ID)
if err != nil || current.CarrierID == 0 {
return false, err
}
item, ok := current.Types[checkType]
if !ok || !item.HasDue || !due.Before(item.Due) {
return false, nil
}
client := c.store.Client()
member := strconv.FormatUint(uint64(card.ID), 10)
pipe := client.TxPipeline()
pipe.HSet(ctx, constants.RedisCardDueKey(card.ID), carrierqueue.FieldDue(checkType), due.UnixMilli())
score := redis.ZAddArgs{XX: true, LT: true, Members: []redis.Z{{Score: float64(due.UnixMilli()), Member: member}}}
pipe.ZAddArgs(ctx, constants.RedisCarrierQueueKey(current.CarrierID), score)
pipe.ZAddArgs(ctx, constants.RedisCarrierPriorityQueueKey(current.CarrierID), score)
_, err = pipe.Exec(ctx)
return err == nil, err
}

View File

@@ -0,0 +1,177 @@
package carrierqueue
import (
"context"
"strconv"
"time"
"github.com/redis/go-redis/v9"
"go.uber.org/zap"
"gorm.io/gorm"
"github.com/break/junhong_cmp_fiber/pkg/constants"
)
// FailureEvent 是一次需要告警的检查失败。
type FailureEvent struct {
CardID uint
ICCID string
CarrierID uint
CheckType string
Category string
Message string
FailCount int
FirstFailed time.Time
At time.Time
}
type alertOp struct {
failure *FailureEvent
recovered bool
cardID uint
checkType string
}
// AlertWriter 异步写卡轮询失败告警:同一卡同一检查类型只保留一条未处理告警,后续失败更新计数;
// 首次成功转为已恢复;同账户窗口内新增告警超阈值时产生账户级告警。写入失败只记日志,不影响轮询。
type AlertWriter struct {
db *gorm.DB
redis *redis.Client
settings *SettingsProvider
logger *zap.Logger
ch chan alertOp
}
// NewAlertWriter 创建告警写入器;调用 Start 后开始消费。
func NewAlertWriter(db *gorm.DB, redisClient *redis.Client, settings *SettingsProvider, logger *zap.Logger) *AlertWriter {
if logger == nil {
logger = zap.NewNop()
}
return &AlertWriter{db: db, redis: redisClient, settings: settings, logger: logger, ch: make(chan alertOp, 2048)}
}
// Start 启动后台写入协程,直到 ctx 结束。
func (w *AlertWriter) Start(ctx context.Context) {
go func() {
for {
select {
case <-ctx.Done():
return
case op := <-w.ch:
w.apply(op)
}
}
}()
}
// ReportFailure 提交一条失败告警(达到阈值后才调用)。
func (w *AlertWriter) ReportFailure(event FailureEvent) {
w.offer(alertOp{failure: &event})
}
// ReportRecovered 提交一次恢复:该卡该检查类型的未处理告警转为已恢复。
func (w *AlertWriter) ReportRecovered(cardID uint, checkType string) {
w.offer(alertOp{recovered: true, cardID: cardID, checkType: checkType})
}
func (w *AlertWriter) offer(op alertOp) {
if w == nil {
return
}
select {
case w.ch <- op:
default:
w.logger.Warn("卡轮询失败告警队列已满,丢弃本次写入", zap.String("metric", "polling.failure_alert.dropped"))
}
}
func (w *AlertWriter) apply(op alertOp) {
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
if op.recovered {
err := w.db.WithContext(ctx).Exec(`UPDATE tb_polling_failure_alert SET status = ?, recovered_at = NOW(), updated_at = NOW()
WHERE alert_level = ? AND card_id = ? AND check_type = ? AND status = ?`,
constants.PollingAlertStatusRecovered, constants.PollingAlertLevelCard, op.cardID, op.checkType, constants.PollingAlertStatusOpen).Error
if err != nil {
w.logger.Warn("卡轮询告警转为已恢复失败", zap.Uint("card_id", op.cardID), zap.String("check_type", op.checkType), zap.Error(err))
}
return
}
event := op.failure
var inserted bool
err := w.db.WithContext(ctx).Raw(`INSERT INTO tb_polling_failure_alert
(alert_level, card_id, iccid, carrier_id, check_type, error_category, fail_count, first_failed_at, last_failed_at, last_error, status)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
ON CONFLICT (alert_level, card_id, carrier_id, check_type) WHERE status = 1
DO UPDATE SET fail_count = EXCLUDED.fail_count, error_category = EXCLUDED.error_category,
last_failed_at = EXCLUDED.last_failed_at, last_error = EXCLUDED.last_error, updated_at = NOW()
RETURNING (xmax = 0)`,
constants.PollingAlertLevelCard, event.CardID, event.ICCID, event.CarrierID, event.CheckType, event.Category, event.FailCount,
nullableTime(event.FirstFailed), event.At, truncate(event.Message, 500), constants.PollingAlertStatusOpen).Row().Scan(&inserted)
if err != nil {
w.logger.Warn("写卡轮询失败告警失败", zap.Uint("card_id", event.CardID), zap.String("check_type", event.CheckType), zap.Error(err))
return
}
if !inserted {
return
}
w.logger.Error("卡轮询持续失败告警", zap.Uint("card_id", event.CardID), zap.String("iccid", event.ICCID),
zap.Uint("carrier_id", event.CarrierID), zap.String("check_type", event.CheckType), zap.String("category", event.Category),
zap.Int("fail_count", event.FailCount), zap.String("metric", "polling.failure_alert.opened"))
w.checkAccountAlert(ctx, event.CarrierID)
}
// checkAccountAlert 同账户窗口内新增卡级告警数超过阈值时,产生(或更新)一条账户级告警。
func (w *AlertWriter) checkAccountAlert(ctx context.Context, carrierID uint) {
settings := w.settings.Current(ctx)
window, limit := settings.AccountAlertWindow, settings.AccountAlertLimit
if window <= 0 || limit <= 0 || w.redis == nil {
return
}
now := time.Now()
key := constants.RedisCarrierAccountAlertKey(carrierID)
member := strconv.FormatInt(now.UnixNano(), 36)
pipe := w.redis.TxPipeline()
pipe.ZAdd(ctx, key, redis.Z{Score: float64(now.UnixMilli()), Member: member})
pipe.ZRemRangeByScore(ctx, key, "-inf", strconv.FormatInt(now.Add(-window).UnixMilli(), 10))
count := pipe.ZCard(ctx, key)
pipe.Expire(ctx, key, window*2)
if _, err := pipe.Exec(ctx); err != nil {
return
}
if int(count.Val()) <= limit {
return
}
var inserted bool
err := w.db.WithContext(ctx).Raw(`INSERT INTO tb_polling_failure_alert
(alert_level, card_id, carrier_id, check_type, error_category, fail_count, first_failed_at, last_failed_at, last_error, status)
VALUES (?, 0, ?, ?, '', ?, ?, ?, ?, ?)
ON CONFLICT (alert_level, card_id, carrier_id, check_type) WHERE status = 1
DO UPDATE SET fail_count = EXCLUDED.fail_count, last_failed_at = EXCLUDED.last_failed_at, updated_at = NOW()
RETURNING (xmax = 0)`,
constants.PollingAlertLevelCarrier, carrierID, constants.PollingAlertCheckTypeAll, count.Val(), now, now,
"窗口内同账户新增失败告警数超过阈值,通常是账户额度、凭证或上游故障,请联系上游处理账户", constants.PollingAlertStatusOpen).Row().Scan(&inserted)
if err != nil {
w.logger.Warn("写账户级轮询告警失败", zap.Uint("carrier_id", carrierID), zap.Error(err))
return
}
if inserted {
w.logger.Error("运营商账户轮询失败告警数超过阈值", zap.Uint("carrier_id", carrierID), zap.Int64("count", count.Val()),
zap.Duration("window", window), zap.String("metric", "polling.failure_alert.carrier_opened"))
}
}
func nullableTime(t time.Time) any {
if t.IsZero() {
return nil
}
return t
}
func truncate(s string, max int) string {
runes := []rune(s)
if len(runes) <= max {
return s
}
return string(runes[:max])
}

View File

@@ -0,0 +1,326 @@
package carrierqueue
import (
"context"
"encoding/json"
stderrors "errors"
"strconv"
"sync"
"time"
"github.com/redis/go-redis/v9"
"go.uber.org/zap"
"gorm.io/gorm"
"github.com/break/junhong_cmp_fiber/internal/model"
"github.com/break/junhong_cmp_fiber/pkg/config"
"github.com/break/junhong_cmp_fiber/pkg/constants"
)
// CardMessage 是卡任务 NSQ 消息体。
type CardMessage struct {
CardID uint `json:"card_id"`
CarrierID uint `json:"carrier_id"`
Token string `json:"token"`
DispatchedAt int64 `json:"dispatched_at"`
}
// Publisher 发布 NSQ 消息。
type Publisher interface {
Publish(topic string, body []byte) error
}
// BudgetProvider 返回账户各查询操作所属限流组本秒剩余名额;limited=false 表示不按 QPS 限制领取。
type BudgetProvider interface {
Budget(ctx context.Context, carrierID uint, ops []string) (groups map[string]int, opGroup map[string]string, limited bool)
}
// CarrierParams 是一个运营商账户生效的调度参数(已合并全局默认)。
type CarrierParams struct {
InflightLimit int
CardMaxInFlight int
PriorityRatio float64
Configured bool // 是否有该账户的配置行
}
// carrierEntry 是分发器缓存的账户信息。
type carrierEntry struct {
id uint
realnameControl bool
}
// Dispatcher 每个分发间隔对每个运营商账户抢分发锁、领取到期卡并发布到账户 topic。多实例安全。
type Dispatcher struct {
db *gorm.DB
store *Store
settings *SettingsProvider
publisher Publisher
budgets BudgetProvider
cfg config.CarrierSchedulerConfig
instanceID string
logger *zap.Logger
mu sync.Mutex
carriers []carrierEntry
params map[uint]model.CarrierPollingParam
loadedAt time.Time
tightened map[uint]time.Time
}
// NewDispatcher 创建分发器。
func NewDispatcher(db *gorm.DB, store *Store, settings *SettingsProvider, publisher Publisher, budgets BudgetProvider,
cfg config.CarrierSchedulerConfig, instanceID string, logger *zap.Logger) *Dispatcher {
if logger == nil {
logger = zap.NewNop()
}
return &Dispatcher{db: db, store: store, settings: settings, publisher: publisher, budgets: budgets, cfg: cfg,
instanceID: instanceID, logger: logger, params: map[uint]model.CarrierPollingParam{}, tightened: map[uint]time.Time{}}
}
// Run 按分发间隔循环,直到 ctx 结束。
func (d *Dispatcher) Run(ctx context.Context) {
interval := d.cfg.DispatchInterval
if interval <= 0 {
interval = time.Second
}
ticker := time.NewTicker(interval)
defer ticker.Stop()
for {
select {
case <-ctx.Done():
return
case <-ticker.C:
d.tick(ctx, interval)
}
}
}
// Ready 判断新调度是否可领取:模式为 carrier 且新结构初始化已完成。
func (d *Dispatcher) Ready(ctx context.Context) bool {
if !d.settings.Current(ctx).CarrierMode() {
return false
}
state, err := d.store.redis.Get(ctx, constants.RedisCarrierSchedulerInitStateKey).Result()
return err == nil && state == InitStateDone
}
func (d *Dispatcher) tick(ctx context.Context, interval time.Duration) {
defer func() {
if r := recover(); r != nil {
d.logger.Error("分发器 panic,已恢复", zap.Any("panic", r))
}
}()
if !d.Ready(ctx) {
return
}
d.store.redis.HSet(ctx, constants.RedisCarrierSchedulerHeartbeatKey, d.instanceID, time.Now().UnixMilli())
d.drainLegacyHints(ctx)
d.refreshCarriers(ctx)
d.mu.Lock()
carriers := append([]carrierEntry(nil), d.carriers...)
d.mu.Unlock()
lockTTL := interval * 9 / 10
if lockTTL <= 0 {
lockTTL = 900 * time.Millisecond
}
for _, carrier := range carriers {
if ctx.Err() != nil {
return
}
ok, err := d.store.redis.SetNX(ctx, constants.RedisCarrierDispatchLockKey(carrier.id), d.instanceID, lockTTL).Result()
if err != nil || !ok {
continue
}
d.dispatchCarrier(ctx, carrier)
}
}
func (d *Dispatcher) dispatchCarrier(ctx context.Context, carrier carrierEntry) {
params := d.ParamsFor(carrier.id)
limit := params.InflightLimit
// 执行上限不得大于订阅进程 max_in_flight 之和,否则领出的消息在 NSQ 中排队到租约过期;不满足时收紧本轮上限并上报。
processes := d.cfg.SubscriberProcesses
if processes <= 0 {
processes = 1
}
if capacity := params.CardMaxInFlight * processes; capacity > 0 && limit > capacity {
limit = capacity
d.reportTightened(carrier.id, params.InflightLimit, capacity)
}
input := ClaimInput{
CarrierID: carrier.id, Lease: d.lease(), InflightLimit: limit, PriorityRatio: params.PriorityRatio,
TokenPrefix: d.instanceID + ":" + strconv.FormatInt(time.Now().UnixNano(), 36),
CountRealname: carrier.realnameControl,
}
if d.budgets != nil {
ops := []string{constants.GatewayOpQueryStatus, constants.GatewayOpQueryFlow, constants.GatewayOpQueryRealname}
if groups, opGroup, limited := d.budgets.Budget(ctx, carrier.id, ops); limited {
input.GroupBudget = groups
input.TypeGroup = map[string]string{
constants.CheckTypeCardStatus: opGroup[constants.GatewayOpQueryStatus],
constants.CheckTypeCarddata: opGroup[constants.GatewayOpQueryFlow],
constants.CheckTypeRealname: opGroup[constants.GatewayOpQueryRealname],
}
}
}
claimed, expired, err := d.store.Claim(ctx, input)
if err != nil {
d.logger.Warn("运营商账户领取失败", zap.Uint("carrier_id", carrier.id), zap.Error(err))
return
}
if expired > 0 {
minute := strconv.FormatInt(time.Now().Unix()/60, 10)
key := constants.RedisCarrierExpiredClaimsKey(carrier.id)
d.store.redis.HIncrBy(ctx, key, minute, expired)
d.store.redis.Expire(ctx, key, 2*time.Hour)
d.logger.Warn("运营商账户处理中标记到期(执行崩溃或超时)", zap.Uint("carrier_id", carrier.id), zap.Int64("count", expired),
zap.String("metric", "polling.carrier.claim_expired"))
}
topic := constants.NSQCardTopic(carrier.id)
for _, item := range claimed {
body, _ := json.Marshal(CardMessage{CardID: item.CardID, CarrierID: carrier.id, Token: item.Token, DispatchedAt: time.Now().UnixMilli()})
if err := d.publisher.Publish(topic, body); err != nil {
// 发布失败:用独立短超时上下文按 token 删除处理中标记并恢复 score;恢复也失败则等租约到期后自然重新领取。
releaseCtx, cancel := context.WithTimeout(context.WithoutCancel(ctx), 2*time.Second)
if releaseErr := d.store.Release(releaseCtx, item.CardID, carrier.id, item.Token); releaseErr != nil {
d.logger.Warn("发布失败后释放处理中标记失败", zap.Uint("card_id", item.CardID), zap.Error(releaseErr))
}
cancel()
d.logger.Warn("发布卡任务失败,已释放", zap.Uint("card_id", item.CardID), zap.String("topic", topic), zap.Error(err))
}
}
}
func (d *Dispatcher) lease() time.Duration {
if d.cfg.Lease > 0 {
return d.cfg.Lease
}
return 5 * time.Minute
}
func (d *Dispatcher) reportTightened(carrierID uint, configured, capacity int) {
d.mu.Lock()
last := d.tightened[carrierID]
if time.Since(last) < 10*time.Minute {
d.mu.Unlock()
return
}
d.tightened[carrierID] = time.Now()
d.mu.Unlock()
d.logger.Warn("账户执行上限大于订阅进程 max_in_flight 之和,已按消费能力收紧本轮领取上限",
zap.Uint("carrier_id", carrierID), zap.Int("inflight_limit", configured), zap.Int("capacity", capacity),
zap.String("metric", "polling.carrier.inflight_tightened"))
}
// refreshCarriers 每 10 秒重新读取运营商账户与调度参数。
func (d *Dispatcher) refreshCarriers(ctx context.Context) {
d.mu.Lock()
fresh := time.Since(d.loadedAt) < 10*time.Second
d.mu.Unlock()
if fresh {
return
}
var carriers []model.Carrier
if err := d.db.WithContext(ctx).Select("id", "realname_control_enabled").Order("id").Find(&carriers).Error; err != nil {
d.logger.Warn("读取运营商账户失败,沿用上一次列表", zap.Error(err))
return
}
var rows []model.CarrierPollingParam
if err := d.db.WithContext(ctx).Find(&rows).Error; err != nil {
d.logger.Warn("读取运营商账户调度参数失败,沿用上一次参数", zap.Error(err))
return
}
entries := make([]carrierEntry, 0, len(carriers))
for _, carrier := range carriers {
entries = append(entries, carrierEntry{id: carrier.ID, realnameControl: carrier.RealnameControlEnabled == 1})
}
params := make(map[uint]model.CarrierPollingParam, len(rows))
for _, row := range rows {
params[row.CarrierID] = row
}
d.mu.Lock()
d.carriers, d.params, d.loadedAt = entries, params, time.Now()
d.mu.Unlock()
}
// CarrierIDs 返回当前缓存的运营商账户 ID(消费者据此订阅)。
func (d *Dispatcher) CarrierIDs(ctx context.Context) []uint {
d.refreshCarriers(ctx)
d.mu.Lock()
defer d.mu.Unlock()
ids := make([]uint, 0, len(d.carriers))
for _, carrier := range d.carriers {
ids = append(ids, carrier.id)
}
return ids
}
// ParamsFor 返回账户生效调度参数(未配置项用全局默认)。
func (d *Dispatcher) ParamsFor(carrierID uint) CarrierParams {
d.mu.Lock()
row, ok := d.params[carrierID]
d.mu.Unlock()
return MergeParams(row, ok, d.cfg)
}
// MergeParams 合并账户配置行与全局默认。
func MergeParams(row model.CarrierPollingParam, configured bool, cfg config.CarrierSchedulerConfig) CarrierParams {
params := CarrierParams{InflightLimit: cfg.DefaultInflightLimit, CardMaxInFlight: cfg.DefaultCardMaxInFlight, Configured: configured}
if params.InflightLimit <= 0 {
params.InflightLimit = 20
}
if params.CardMaxInFlight <= 0 {
params.CardMaxInFlight = 20
}
if !configured {
return params
}
if row.InflightLimit != nil && *row.InflightLimit > 0 {
params.InflightLimit = *row.InflightLimit
}
if row.CardMaxInFlight != nil && *row.CardMaxInFlight > 0 {
params.CardMaxInFlight = *row.CardMaxInFlight
}
if row.PriorityMaxRatio != nil && *row.PriorityMaxRatio > 0 {
params.PriorityRatio = *row.PriorityMaxRatio
}
return params
}
// drainLegacyHints 消费旧优先提示通道与手动触发队列:优先提示放入账户优先队列,手动触发在普通队列中立即到期。
func (d *Dispatcher) drainLegacyHints(ctx context.Context) {
for _, checkType := range constants.AllCheckTypes() {
taskType := constants.TaskTypeFromCheckType(checkType)
d.drainList(ctx, constants.RedisPollingPriorityQueueKey(taskType), checkType, false)
d.drainList(ctx, constants.RedisPollingManualQueueKey(taskType), checkType, true)
}
}
func (d *Dispatcher) drainList(ctx context.Context, key, checkType string, normal bool) {
for i := 0; i < 500; i++ {
raw, err := d.store.redis.LPop(ctx, key).Result()
if err != nil {
if !stderrors.Is(err, redis.Nil) {
d.logger.Warn("读取旧提示队列失败", zap.String("key", key), zap.Error(err))
}
return
}
cardID, convErr := strconv.ParseUint(raw, 10, 64)
if convErr != nil {
continue
}
due, dueErr := d.store.ReadDue(ctx, uint(cardID))
if dueErr != nil || due.CarrierID == 0 {
continue
}
if _, err := d.store.RequestCheck(ctx, CheckRequest{CardID: uint(cardID), CarrierID: due.CarrierID, CheckTypes: []string{checkType}, Normal: normal}); err != nil {
d.logger.Warn("写入账户队列失败", zap.Uint64("card_id", cardID), zap.Error(err))
}
}
}
// 新结构初始化状态(constants.RedisCarrierSchedulerInitStateKey)。
const (
InitStateRunning = "running"
InitStateDone = "done"
)

View File

@@ -0,0 +1,254 @@
package carrierqueue
import "github.com/redis/go-redis/v9"
// 卡到期 Hash 字段前缀(见 constants.RedisCardDueKey)。
const (
fieldDue = "due:" // 各检查类型下次到期(毫秒)
fieldOK = "ok:" // 最近一次成功(毫秒),用于冷却判断
fieldFail = "fail:" // 连续失败次数
fieldFirstFail = "ffail:" // 首次失败(毫秒)
fieldError = "err:" // 最近错误(类别|信息)
fieldConfirm = "cfm:" // 待执行的延迟确认序列(秒,逗号分隔)
fieldExpected = "exp:" // 延迟确认的期望结果(如 offline / online),满足时取消后续确认
fieldPriority = "prio" // 优先标记:1 表示应在优先队列(执行中收到请求检查时由收尾移入优先队列)
fieldCarrier = "carrier"
)
// claimLua 账户领取(分发器抢到账户分发锁后执行)。
//
// KEYS[1] 优先队列;KEYS[2] 普通队列;KEYS[3] 执行中计数。
// ARGV:now_ms、lease_ms、执行上限、优先占比(0 表示不限)、token 前缀、
// 限流组本秒剩余("g=n;g=n",空串表示不按 QPS 限制)、检查类型所属限流组("type=g;...")、是否计入实名(1/0)。
// 返回 {本轮清理的过期执行数, card_id, token, 是否来自优先(1/0), ...}。
const claimLua = `
local now = tonumber(ARGV[1])
local lease = tonumber(ARGV[2])
local limit = tonumber(ARGV[3])
local ratio = tonumber(ARGV[4])
local prefix = ARGV[5]
local unlimited = ARGV[6] == ''
local countRealname = ARGV[8] == '1'
local expired = redis.call('ZREMRANGEBYSCORE', KEYS[3], '-inf', now)
local result = {expired}
local capacity = limit - redis.call('ZCARD', KEYS[3])
if capacity <= 0 then
return result
end
local budget = {}
for g, n in string.gmatch(ARGV[6], '([^=;]+)=(%-?%d+)') do budget[g] = tonumber(n) end
local typeGroup = {}
for t, g in string.gmatch(ARGV[7], '([^=;]+)=([^;]+)') do typeGroup[t] = g end
local gatewayTypes = {'card_status', 'carddata', 'realname'}
local used = {}
local taken = 0
local stop = false
local function tryClaim(cardId, fromPriority, qkey)
local dueKey = 'polling:card:' .. cardId .. ':due'
local need = {}
if not unlimited then
for _, t in ipairs(gatewayTypes) do
if t ~= 'realname' or countRealname then
local d = tonumber(redis.call('HGET', dueKey, 'due:' .. t) or '-1')
if d >= 0 and d <= now then
local g = typeGroup[t]
if g then need[g] = (need[g] or 0) + 1 end
end
end
end
for g, n in pairs(need) do
if budget[g] ~= nil and (used[g] or 0) + n > budget[g] then
stop = true
return false
end
end
end
local claimKey = 'polling:card:' .. cardId .. ':claim'
local token = prefix .. ':' .. cardId
local ok = redis.call('SET', claimKey, token, 'NX', 'PX', lease)
if not ok then
local ttl = redis.call('PTTL', claimKey)
if ttl < 0 then ttl = lease end
redis.call('ZADD', qkey, 'XX', now + ttl, cardId)
return false
end
for g, n in pairs(need) do used[g] = (used[g] or 0) + n end
redis.call('ZADD', qkey, 'XX', now + lease, cardId)
redis.call('ZADD', KEYS[3], now + lease, token)
if fromPriority then
redis.call('HDEL', dueKey, 'prio')
end
taken = taken + 1
table.insert(result, cardId)
table.insert(result, token)
table.insert(result, fromPriority and '1' or '0')
return true
end
local function drain(qkey, fromPriority, quota)
if quota <= 0 or stop then return end
local ids = redis.call('ZRANGEBYSCORE', qkey, '-inf', now, 'LIMIT', 0, quota * 2 + 10)
for _, id in ipairs(ids) do
if stop or quota <= 0 or taken >= capacity then return end
if tryClaim(id, fromPriority, qkey) then quota = quota - 1 end
end
end
local priorityQuota = capacity
if ratio > 0 then
priorityQuota = math.floor(capacity * ratio)
if priorityQuota < 1 then priorityQuota = 1 end
end
drain(KEYS[1], true, priorityQuota)
drain(KEYS[2], false, capacity - taken)
drain(KEYS[1], true, capacity - taken)
return result
`
// renewLua 续期:token 与执行权一致时延长租约与执行权,并推后 score 与执行中计数到期时间。
//
// KEYS[1] claim;KEYS[2] exec;KEYS[3] 执行中计数;KEYS[4] 优先队列;KEYS[5] 普通队列。
// ARGV:token、执行权值、lease_ms、exec_ttl_ms、now_ms、card_id。返回 1 成功,0 token 或执行权已不一致。
const renewLua = `
if redis.call('GET', KEYS[1]) ~= ARGV[1] then return 0 end
if redis.call('GET', KEYS[2]) ~= ARGV[2] then return 0 end
local lease = tonumber(ARGV[3])
local now = tonumber(ARGV[5])
redis.call('PEXPIRE', KEYS[1], lease)
redis.call('PEXPIRE', KEYS[2], tonumber(ARGV[4]))
redis.call('ZADD', KEYS[3], 'XX', now + lease, ARGV[1])
redis.call('ZADD', KEYS[4], 'XX', now + lease, ARGV[6])
redis.call('ZADD', KEYS[5], 'XX', now + lease, ARGV[6])
return 1
`
// acquireExecLua 抢执行权:claim 等于 token 时 SET exec NX PX。返回 1 抢到,0 claim 不一致,-1 已有执行者。
//
// KEYS[1] claim;KEYS[2] exec。ARGV:token、执行权值、exec_ttl_ms。
const acquireExecLua = `
if redis.call('GET', KEYS[1]) ~= ARGV[1] then return 0 end
if redis.call('SET', KEYS[2], ARGV[2], 'NX', 'PX', tonumber(ARGV[3])) then return 1 end
return -1
`
// finishLua 收尾:写回业务结果;token 一致时删除处理中标记与执行权、移出执行中计数,
// 按优先标记把卡放入优先或普通队列,score = 各检查类型最早到期。卡已不在任一队列(已移出轮询)时不重新加入。
//
// KEYS[1] due;KEYS[2] claim;KEYS[3] exec;KEYS[4] 执行中计数;KEYS[5] 优先队列;KEYS[6] 普通队列。
// ARGV:token、card_id、now_ms、set 字段对数 n、字段1、值1、…、删除字段…。
// 返回 1 正常收尾,0 token 不一致(只写了业务结果)。
const finishLua = `
local n = tonumber(ARGV[4])
local idx = 5
for i = 1, n do
redis.call('HSET', KEYS[1], ARGV[idx], ARGV[idx + 1])
idx = idx + 2
end
while idx <= #ARGV do
redis.call('HDEL', KEYS[1], ARGV[idx])
idx = idx + 1
end
if redis.call('GET', KEYS[2]) ~= ARGV[1] then return 0 end
redis.call('DEL', KEYS[2])
redis.call('DEL', KEYS[3])
redis.call('ZREM', KEYS[4], ARGV[1])
local cardId = ARGV[2]
local inPriority = redis.call('ZSCORE', KEYS[5], cardId)
local inQueue = redis.call('ZSCORE', KEYS[6], cardId)
if not inPriority and not inQueue then return 1 end
local fields = redis.call('HGETALL', KEYS[1])
local nextDue = nil
for i = 1, #fields, 2 do
if string.sub(fields[i], 1, 4) == 'due:' then
local v = tonumber(fields[i + 1])
if v and (nextDue == nil or v < nextDue) then nextDue = v end
end
end
if nextDue == nil then nextDue = tonumber(ARGV[3]) + 3600000 end
if redis.call('HGET', KEYS[1], 'prio') == '1' then
redis.call('ZREM', KEYS[6], cardId)
redis.call('ZADD', KEYS[5], nextDue, cardId)
else
redis.call('ZREM', KEYS[5], cardId)
redis.call('ZADD', KEYS[6], nextDue, cardId)
end
return 1
`
// requestCheckLua 请求检查:把指定检查类型设为立即到期并放入优先队列;卡执行中时只打优先标记,收尾后立即再执行。
// 只作用于仍在轮询中的卡(在优先或普通队列中)。冷却期内(最近成功距今未超过冷却)的类型不请求。
//
// KEYS[1] due;KEYS[2] 优先队列;KEYS[3] 普通队列;KEYS[4] claim。
// ARGV:now_ms、card_id、cooldown_ms(0 不冷却)、确认延迟(秒,逗号分隔,可空)、期望结果(可空)、
// 是否放入优先队列(1/0;0 用于手动触发:只把到期设为现在并在普通队列中立即到期)、类型…。
// 返回本次实际请求的类型数;卡不在轮询中返回 -1。
const requestCheckLua = `
local now = tonumber(ARGV[1])
local cardId = ARGV[2]
local cooldown = tonumber(ARGV[3])
local inPriority = redis.call('ZSCORE', KEYS[2], cardId)
local inQueue = redis.call('ZSCORE', KEYS[3], cardId)
if not inPriority and not inQueue then return -1 end
local requested = 0
local toPriority = ARGV[6] == '1'
for i = 7, #ARGV do
local t = ARGV[i]
local skip = false
if cooldown > 0 then
local ok = tonumber(redis.call('HGET', KEYS[1], 'ok:' .. t) or '0')
if ok > 0 and now - ok < cooldown then skip = true end
end
if not skip then
redis.call('HSET', KEYS[1], 'due:' .. t, now)
if ARGV[4] ~= '' then
redis.call('HSET', KEYS[1], 'cfm:' .. t, ARGV[4])
if ARGV[5] ~= '' then redis.call('HSET', KEYS[1], 'exp:' .. t, ARGV[5]) else redis.call('HDEL', KEYS[1], 'exp:' .. t) end
end
requested = requested + 1
end
end
if requested == 0 then return 0 end
if not toPriority then
if redis.call('EXISTS', KEYS[4]) == 1 then return requested end
if inQueue then redis.call('ZADD', KEYS[3], now, cardId) else redis.call('ZADD', KEYS[2], now, cardId) end
return requested
end
redis.call('HSET', KEYS[1], 'prio', '1')
if redis.call('EXISTS', KEYS[4]) == 1 then return requested end
redis.call('ZREM', KEYS[3], cardId)
redis.call('ZADD', KEYS[2], now, cardId)
return requested
`
// removeLua 把卡移出所属账户队列(关闭轮询、删除卡等);执行中的卡收尾时因不在队列中而不会被重新加入。
// KEYS[1] 优先队列;KEYS[2] 普通队列;KEYS[3] due。ARGV:card_id。
const removeLua = `
redis.call('ZREM', KEYS[1], ARGV[1])
redis.call('ZREM', KEYS[2], ARGV[1])
redis.call('DEL', KEYS[3])
return 1
`
// releaseLua 发布失败时按 token 删除处理中标记、移出执行中计数,并把 score 恢复为当前时间。
// KEYS[1] claim;KEYS[2] 执行中计数;KEYS[3] 优先队列;KEYS[4] 普通队列。ARGV:token、card_id、now_ms。
const releaseLua = `
if redis.call('GET', KEYS[1]) ~= ARGV[1] then return 0 end
redis.call('DEL', KEYS[1])
redis.call('ZREM', KEYS[2], ARGV[1])
redis.call('ZADD', KEYS[3], 'XX', tonumber(ARGV[3]), ARGV[2])
redis.call('ZADD', KEYS[4], 'XX', tonumber(ARGV[3]), ARGV[2])
return 1
`
var (
claimScript = redis.NewScript(claimLua)
renewScript = redis.NewScript(renewLua)
acquireExecScript = redis.NewScript(acquireExecLua)
finishScript = redis.NewScript(finishLua)
requestCheckScript = redis.NewScript(requestCheckLua)
removeScript = redis.NewScript(removeLua)
releaseScript = redis.NewScript(releaseLua)
)

View File

@@ -0,0 +1,392 @@
// Package carrierqueue 实现按运营商账户调度的卡轮询结构:账户延时队列、卡到期信息、领取 / 续期 / 收尾 / 请求检查 Lua、
// 分发器、初始化与受控系统配置。Redis 是唯一权威状态,NSQ 只做传输(add-carrier-polling-scheduler)。
package carrierqueue
import (
"context"
"encoding/json"
stderrors "errors"
"strconv"
"sync"
"time"
"github.com/redis/go-redis/v9"
"go.uber.org/zap"
"gorm.io/gorm"
"github.com/break/junhong_cmp_fiber/internal/infrastructure/systemconfig"
"github.com/break/junhong_cmp_fiber/pkg/constants"
)
// settingsTTL 是调度参数在进程内的缓存时长;后台修改数秒内生效。
const settingsTTL = 5 * time.Second
// RegisterConfigDefinitions 注册按运营商账户调度的受控系统配置定义(含 JSON 结构校验)。
func RegisterConfigDefinitions(registry *systemconfig.Registry, logger *zap.Logger) {
if registry == nil {
return
}
minOne, maxSeconds, maxCount := int64(1), int64(86400*7), int64(100000)
definitions := []systemconfig.Definition{
{Key: constants.SystemConfigPollingSchedulerMode, Module: constants.SystemConfigModulePolling, ValueType: constants.SystemConfigTypeString,
DefaultValue: constants.PollingSchedulerModeLegacy, Description: "卡轮询调度模式:legacy 旧分片调度,carrier 按运营商账户调度(整体切换)",
EnumValues: []string{constants.PollingSchedulerModeLegacy, constants.PollingSchedulerModeCarrier}},
{Key: constants.SystemConfigPollingStepOrder, Module: constants.SystemConfigModulePolling, ValueType: constants.SystemConfigTypeJSON,
DefaultValue: `["card_status","carddata","realname","package"]`, Description: "卡任务步骤顺序", Validator: validateStepOrder},
{Key: constants.SystemConfigPollingStepTimeout, Module: constants.SystemConfigModulePolling, ValueType: constants.SystemConfigTypeJSON,
DefaultValue: `{"card_status":15,"carddata":15,"realname":15,"package":20}`, Description: "各检查类型单步超时(秒)", Validator: validateStepTimeout},
{Key: constants.SystemConfigPollingTaskTimeoutMargin, Module: constants.SystemConfigModulePolling, ValueType: constants.SystemConfigTypeInt,
DefaultValue: "10", Description: "卡任务总超时余量(秒)", Min: &minOne, Max: &maxSeconds},
{Key: constants.SystemConfigPollingSkipPackageAfterFlow, Module: constants.SystemConfigModulePolling, ValueType: constants.SystemConfigTypeBool,
DefaultValue: "true", Description: "同轮流量步骤已触发停复机评估时跳过套餐步骤"},
{Key: constants.SystemConfigPollingFailureBackoff, Module: constants.SystemConfigModulePolling, ValueType: constants.SystemConfigTypeJSON,
DefaultValue: `{"default":[60,300,900,1800,3600]}`, Description: "失败退避序列(秒,递增),可按检查类型覆盖 default", Validator: validateBackoff},
{Key: constants.SystemConfigPollingAlertFailCount, Module: constants.SystemConfigModulePolling, ValueType: constants.SystemConfigTypeJSON,
DefaultValue: `{"default":5}`, Description: "连续失败告警阈值,可按检查类型覆盖 default", Validator: validateAlertCount},
{Key: constants.SystemConfigPollingAlertFailDuration, Module: constants.SystemConfigModulePolling, ValueType: constants.SystemConfigTypeInt,
DefaultValue: "21600", Description: "持续失败时长告警阈值(秒)", Min: &minOne, Max: &maxSeconds},
{Key: constants.SystemConfigPollingImmediateAlert, Module: constants.SystemConfigModulePolling, ValueType: constants.SystemConfigTypeJSON,
DefaultValue: `["account_config"]`, Description: "首次出现即告警的错误类别", Validator: validateCategories},
{Key: constants.SystemConfigPollingAccountAlertWindow, Module: constants.SystemConfigModulePolling, ValueType: constants.SystemConfigTypeInt,
DefaultValue: "600", Description: "账户级告警统计窗口(秒)", Min: &minOne, Max: &maxSeconds},
{Key: constants.SystemConfigPollingAccountAlertThreshold, Module: constants.SystemConfigModulePolling, ValueType: constants.SystemConfigTypeInt,
DefaultValue: "50", Description: "账户级告警阈值:窗口内同账户新增失败告警数", Min: &minOne, Max: &maxCount},
}
for _, definition := range definitions {
if _, exists := registry.Get(definition.Key); exists {
continue
}
if err := registry.Register(definition); err != nil && logger != nil {
logger.Error("注册按运营商账户调度系统配置失败,将使用代码默认值", zap.String("config_key", definition.Key), zap.Error(err))
}
}
}
func validCheckType(value string) bool {
switch value {
case constants.CheckTypeCardStatus, constants.CheckTypeCarddata, constants.CheckTypeRealname, constants.CheckTypePackage:
return true
}
return false
}
func validateStepOrder(raw string) error {
var steps []string
if err := json.Unmarshal([]byte(raw), &steps); err != nil {
return stderrors.New("步骤顺序必须是字符串数组")
}
if len(steps) == 0 {
return stderrors.New("步骤顺序不能为空")
}
seen := map[string]bool{}
for _, step := range steps {
if !validCheckType(step) {
return stderrors.New("步骤只能是 card_status、carddata、realname、package")
}
if seen[step] {
return stderrors.New("步骤不能重复")
}
seen[step] = true
}
return nil
}
func validateStepTimeout(raw string) error {
var timeouts map[string]int
if err := json.Unmarshal([]byte(raw), &timeouts); err != nil {
return stderrors.New("单步超时必须是 {检查类型: 秒} 对象")
}
for key, value := range timeouts {
if !validCheckType(key) {
return stderrors.New("单步超时的检查类型非法")
}
if value <= 0 || value > 600 {
return stderrors.New("单步超时必须在 1~600 秒之间")
}
}
return nil
}
func validateBackoff(raw string) error {
var backoff map[string][]int
if err := json.Unmarshal([]byte(raw), &backoff); err != nil {
return stderrors.New("退避序列必须是 {default 或检查类型: [秒...]} 对象")
}
if len(backoff["default"]) == 0 {
return stderrors.New("退避序列必须包含非空的 default")
}
for key, values := range backoff {
if key != "default" && !validCheckType(key) {
return stderrors.New("退避序列的检查类型非法")
}
for index, value := range values {
if value <= 0 {
return stderrors.New("退避间隔必须为正数")
}
if index > 0 && value <= values[index-1] {
return stderrors.New("退避序列必须递增")
}
}
}
return nil
}
func validateAlertCount(raw string) error {
var counts map[string]int
if err := json.Unmarshal([]byte(raw), &counts); err != nil {
return stderrors.New("告警阈值必须是 {default 或检查类型: 次数} 对象")
}
if counts["default"] <= 0 {
return stderrors.New("告警阈值必须包含正数 default")
}
for key, value := range counts {
if key != "default" && !validCheckType(key) {
return stderrors.New("告警阈值的检查类型非法")
}
if value <= 0 {
return stderrors.New("告警阈值必须为正数")
}
}
return nil
}
func validateCategories(raw string) error {
var categories []string
if err := json.Unmarshal([]byte(raw), &categories); err != nil {
return stderrors.New("错误类别必须是字符串数组")
}
return nil
}
// Settings 是按运营商账户调度的运行参数快照。
type Settings struct {
Mode string
StepOrder []string
StepTimeouts map[string]time.Duration
TaskTimeoutMargin time.Duration
SkipPackageAfterFlow bool
Backoff map[string][]time.Duration
AlertFailCount map[string]int
AlertFailDuration time.Duration
ImmediateAlert map[string]bool
AccountAlertWindow time.Duration
AccountAlertLimit int
}
// CarrierMode 判断当前是否为按运营商账户调度。
func (s Settings) CarrierMode() bool {
return s.Mode == constants.PollingSchedulerModeCarrier
}
// StepTimeout 返回检查类型单步超时;未配置时 15 秒。
func (s Settings) StepTimeout(checkType string) time.Duration {
if value, ok := s.StepTimeouts[checkType]; ok && value > 0 {
return value
}
return 15 * time.Second
}
// TaskTimeout 返回卡任务总超时 = 各步骤超时之和 + 余量。
func (s Settings) TaskTimeout() time.Duration {
total := s.TaskTimeoutMargin
for _, step := range s.StepOrder {
total += s.StepTimeout(step)
}
return total
}
// BackoffFor 返回检查类型第 n 次连续失败(n 从 1 起)的退避间隔,不超过 maxInterval(> 0 时)。
func (s Settings) BackoffFor(checkType string, failCount int, maxInterval time.Duration) time.Duration {
sequence := s.Backoff[checkType]
if len(sequence) == 0 {
sequence = s.Backoff["default"]
}
if len(sequence) == 0 {
sequence = []time.Duration{time.Minute}
}
index := failCount - 1
if index < 0 {
index = 0
}
if index >= len(sequence) {
index = len(sequence) - 1
}
delay := sequence[index]
if maxInterval > 0 && delay > maxInterval {
delay = maxInterval
}
return delay
}
// AlertThreshold 返回检查类型连续失败告警阈值。
func (s Settings) AlertThreshold(checkType string) int {
if value, ok := s.AlertFailCount[checkType]; ok && value > 0 {
return value
}
if value := s.AlertFailCount["default"]; value > 0 {
return value
}
return 5
}
// DefaultSettings 是配置读取失败时的安全默认值(旧分片调度)。
func DefaultSettings() Settings {
return Settings{
Mode: constants.PollingSchedulerModeLegacy,
StepOrder: constants.AllCheckTypes(),
StepTimeouts: map[string]time.Duration{
constants.CheckTypeCardStatus: 15 * time.Second, constants.CheckTypeCarddata: 15 * time.Second,
constants.CheckTypeRealname: 15 * time.Second, constants.CheckTypePackage: 20 * time.Second,
},
TaskTimeoutMargin: 10 * time.Second, SkipPackageAfterFlow: true,
Backoff: map[string][]time.Duration{"default": {time.Minute, 5 * time.Minute, 15 * time.Minute, 30 * time.Minute, time.Hour}},
AlertFailCount: map[string]int{"default": 5}, AlertFailDuration: 6 * time.Hour,
ImmediateAlert: map[string]bool{constants.GatewayClassAccountConfig: true},
AccountAlertWindow: 10 * time.Minute, AccountAlertLimit: 50,
}
}
// ConfigReader 读取已注册的受控系统配置。
type ConfigReader interface {
Get(ctx context.Context, key string) (string, error)
}
// SettingsProvider 在进程内缓存调度参数,超过 TTL 重新读取。
type SettingsProvider struct {
reader ConfigReader
logger *zap.Logger
mu sync.Mutex
cached Settings
loaded time.Time
}
// NewSettingsProvider 创建调度参数提供者;reader 为 nil 时始终返回默认值。
func NewSettingsProvider(reader ConfigReader, logger *zap.Logger) *SettingsProvider {
if logger == nil {
logger = zap.NewNop()
}
return &SettingsProvider{reader: reader, logger: logger, cached: DefaultSettings()}
}
// Current 返回当前调度参数;读取失败时保留上一次成功值。
func (p *SettingsProvider) Current(ctx context.Context) Settings {
if p == nil {
return DefaultSettings()
}
p.mu.Lock()
defer p.mu.Unlock()
if p.reader == nil || time.Since(p.loaded) < settingsTTL {
return p.cached
}
next, err := p.load(ctx)
p.loaded = time.Now()
if err != nil {
p.logger.Warn("读取按运营商账户调度配置失败,沿用上一次配置", zap.Error(err))
return p.cached
}
p.cached = next
return next
}
// Invalidate 使缓存立即失效(后台保存后调用)。
func (p *SettingsProvider) Invalidate() {
if p == nil {
return
}
p.mu.Lock()
p.loaded = time.Time{}
p.mu.Unlock()
}
func (p *SettingsProvider) load(ctx context.Context) (Settings, error) {
s := DefaultSettings()
get := func(key string) (string, error) { return p.reader.Get(ctx, key) }
raw, err := get(constants.SystemConfigPollingSchedulerMode)
if err != nil {
return s, err
}
s.Mode = raw
if raw, err = get(constants.SystemConfigPollingStepOrder); err != nil {
return s, err
}
if err := json.Unmarshal([]byte(raw), &s.StepOrder); err != nil || validateStepOrder(raw) != nil {
return s, stderrors.New("步骤顺序配置非法")
}
if raw, err = get(constants.SystemConfigPollingStepTimeout); err != nil {
return s, err
}
var timeouts map[string]int
if err := json.Unmarshal([]byte(raw), &timeouts); err != nil {
return s, stderrors.New("单步超时配置非法")
}
for key, value := range timeouts {
s.StepTimeouts[key] = time.Duration(value) * time.Second
}
if raw, err = get(constants.SystemConfigPollingFailureBackoff); err != nil {
return s, err
}
var backoff map[string][]int
if err := json.Unmarshal([]byte(raw), &backoff); err != nil {
return s, stderrors.New("退避序列配置非法")
}
s.Backoff = map[string][]time.Duration{}
for key, values := range backoff {
for _, value := range values {
s.Backoff[key] = append(s.Backoff[key], time.Duration(value)*time.Second)
}
}
if raw, err = get(constants.SystemConfigPollingAlertFailCount); err != nil {
return s, err
}
if err := json.Unmarshal([]byte(raw), &s.AlertFailCount); err != nil {
return s, stderrors.New("告警阈值配置非法")
}
if raw, err = get(constants.SystemConfigPollingImmediateAlert); err != nil {
return s, err
}
var categories []string
if err := json.Unmarshal([]byte(raw), &categories); err != nil {
return s, stderrors.New("首次告警类别配置非法")
}
s.ImmediateAlert = map[string]bool{}
for _, category := range categories {
s.ImmediateAlert[category] = true
}
if raw, err = get(constants.SystemConfigPollingSkipPackageAfterFlow); err != nil {
return s, err
}
s.SkipPackageAfterFlow = raw == "true"
ints := []struct {
key string
set func(int)
}{
{constants.SystemConfigPollingTaskTimeoutMargin, func(v int) { s.TaskTimeoutMargin = time.Duration(v) * time.Second }},
{constants.SystemConfigPollingAlertFailDuration, func(v int) { s.AlertFailDuration = time.Duration(v) * time.Second }},
{constants.SystemConfigPollingAccountAlertWindow, func(v int) { s.AccountAlertWindow = time.Duration(v) * time.Second }},
{constants.SystemConfigPollingAccountAlertThreshold, func(v int) { s.AccountAlertLimit = v }},
}
for _, item := range ints {
raw, err := get(item.key)
if err != nil {
return s, err
}
value, convErr := strconv.Atoi(raw)
if convErr != nil {
return s, stderrors.New("调度整数配置非法")
}
item.set(value)
}
return s, nil
}
// NewDefaultSettingsProvider 用独立注册表与读取器创建调度参数提供者(API 与 Worker 共用同一份数据库与缓存数据)。
func NewDefaultSettingsProvider(db *gorm.DB, redisClient *redis.Client, logger *zap.Logger) *SettingsProvider {
registry := systemconfig.NewRegistry()
RegisterConfigDefinitions(registry, logger)
var cache systemconfig.Cache
if redisClient != nil {
cache = systemconfig.NewRedisCache(redisClient)
}
reader := systemconfig.NewReader(db, registry, cache, systemconfig.NewLogAlertSink(logger))
return NewSettingsProvider(reader, logger)
}

View File

@@ -0,0 +1,284 @@
package carrierqueue
import (
"context"
stderrors "errors"
"fmt"
"strconv"
"strings"
"time"
"github.com/redis/go-redis/v9"
"github.com/break/junhong_cmp_fiber/pkg/constants"
)
// Store 封装按运营商账户调度的 Redis 结构与 Lua 原子操作。
type Store struct {
redis *redis.Client
}
// NewStore 创建账户队列存储。
func NewStore(redisClient *redis.Client) *Store {
return &Store{redis: redisClient}
}
// Client 返回底层 Redis 客户端。
func (s *Store) Client() *redis.Client {
return s.redis
}
// ClaimInput 是一次账户领取的参数。
type ClaimInput struct {
CarrierID uint
Lease time.Duration
InflightLimit int
PriorityRatio float64 // 0 表示不限
TokenPrefix string // 本轮派发 token 前缀(实例 + 轮次),token = 前缀:card_id
GroupBudget map[string]int // 限流组本秒剩余;nil 表示不按 QPS 限制
TypeGroup map[string]string // 检查类型 → 限流组
CountRealname bool // 预计调用数是否计入实名(账户未开实名管控时不计)
}
// Claimed 是一张被领取的卡。
type Claimed struct {
CardID uint
Token string
FromPriority bool
}
// Claim 执行一次账户领取,返回领取到的卡与本轮清理的过期执行数(处理中崩溃或超时次数)。
func (s *Store) Claim(ctx context.Context, input ClaimInput) ([]Claimed, int64, error) {
budget := ""
if input.GroupBudget != nil {
parts := make([]string, 0, len(input.GroupBudget))
for group, remaining := range input.GroupBudget {
parts = append(parts, group+"="+strconv.Itoa(remaining))
}
budget = strings.Join(parts, ";")
if budget == "" {
budget = "none=0"
}
}
typeParts := make([]string, 0, len(input.TypeGroup))
for checkType, group := range input.TypeGroup {
typeParts = append(typeParts, checkType+"="+group)
}
countRealname := "0"
if input.CountRealname {
countRealname = "1"
}
raw, err := claimScript.Run(ctx, s.redis,
[]string{constants.RedisCarrierPriorityQueueKey(input.CarrierID), constants.RedisCarrierQueueKey(input.CarrierID), constants.RedisCarrierRunningKey(input.CarrierID)},
time.Now().UnixMilli(), input.Lease.Milliseconds(), input.InflightLimit, strconv.FormatFloat(input.PriorityRatio, 'f', 3, 64),
input.TokenPrefix, budget, strings.Join(typeParts, ";"), countRealname,
).Slice()
if err != nil {
return nil, 0, err
}
if len(raw) == 0 {
return nil, 0, nil
}
expired, _ := raw[0].(int64)
claimed := make([]Claimed, 0, (len(raw)-1)/3)
for index := 1; index+2 < len(raw); index += 3 {
cardID, _ := strconv.ParseUint(fmt.Sprint(raw[index]), 10, 64)
claimed = append(claimed, Claimed{CardID: uint(cardID), Token: fmt.Sprint(raw[index+1]), FromPriority: fmt.Sprint(raw[index+2]) == "1"})
}
return claimed, expired, nil
}
// AcquireExec 抢执行权:返回 1 抢到,0 处理中标记已不是该 token(过期或重复投递),-1 该 token 已有执行者。
func (s *Store) AcquireExec(ctx context.Context, cardID uint, token, execValue string, ttl time.Duration) (int64, error) {
return acquireExecScript.Run(ctx, s.redis, []string{constants.RedisCardClaimKey(cardID), constants.RedisCardExecKey(cardID)},
token, execValue, ttl.Milliseconds()).Int64()
}
// Renew 续期租约与执行权;返回 false 表示 token 或执行权已不一致,调用方必须停止后续步骤。
func (s *Store) Renew(ctx context.Context, cardID, carrierID uint, token, execValue string, lease, execTTL time.Duration) (bool, error) {
result, err := renewScript.Run(ctx, s.redis,
[]string{constants.RedisCardClaimKey(cardID), constants.RedisCardExecKey(cardID), constants.RedisCarrierRunningKey(carrierID),
constants.RedisCarrierPriorityQueueKey(carrierID), constants.RedisCarrierQueueKey(carrierID)},
token, execValue, lease.Milliseconds(), execTTL.Milliseconds(), time.Now().UnixMilli(), cardID).Int64()
return result == 1, err
}
// Finish 收尾:写回业务结果(set 字段与删除字段),token 一致时释放处理中标记并按最早到期设置 score。
// 返回 false 表示 token 已不一致,只写入了业务结果。
func (s *Store) Finish(ctx context.Context, cardID, carrierID uint, token string, set map[string]string, del []string) (bool, error) {
args := []any{token, cardID, time.Now().UnixMilli(), len(set)}
for field, value := range set {
args = append(args, field, value)
}
for _, field := range del {
args = append(args, field)
}
result, err := finishScript.Run(ctx, s.redis,
[]string{constants.RedisCardDueKey(cardID), constants.RedisCardClaimKey(cardID), constants.RedisCardExecKey(cardID),
constants.RedisCarrierRunningKey(carrierID), constants.RedisCarrierPriorityQueueKey(carrierID), constants.RedisCarrierQueueKey(carrierID)},
args...).Int64()
return result == 1, err
}
// Release 发布失败时按 token 删除处理中标记并把 score 恢复为当前时间。
func (s *Store) Release(ctx context.Context, cardID, carrierID uint, token string) error {
return releaseScript.Run(ctx, s.redis,
[]string{constants.RedisCardClaimKey(cardID), constants.RedisCarrierRunningKey(carrierID),
constants.RedisCarrierPriorityQueueKey(carrierID), constants.RedisCarrierQueueKey(carrierID)},
token, cardID, time.Now().UnixMilli()).Err()
}
// CheckRequest 是一次请求检查。
type CheckRequest struct {
CardID uint
CarrierID uint
CheckTypes []string
Cooldown time.Duration
ConfirmDelays []time.Duration
Expected string // 延迟确认的期望结果(card_status 为 online / offline),满足时取消后续确认
// Normal 为 true 时不放入优先队列:只把到期设为现在、在普通队列中立即到期(手动触发走普通通道)。
Normal bool
}
// RequestCheck 把卡的指定检查类型设为立即到期放入优先队列;返回实际请求的类型数,卡不在轮询中返回 -1。
func (s *Store) RequestCheck(ctx context.Context, request CheckRequest) (int64, error) {
if len(request.CheckTypes) == 0 {
return 0, nil
}
confirm := make([]string, 0, len(request.ConfirmDelays))
for _, delay := range request.ConfirmDelays {
confirm = append(confirm, strconv.FormatInt(int64(delay/time.Second), 10))
}
toPriority := "1"
if request.Normal {
toPriority = "0"
}
args := []any{time.Now().UnixMilli(), request.CardID, request.Cooldown.Milliseconds(), strings.Join(confirm, ","), request.Expected, toPriority}
for _, checkType := range request.CheckTypes {
args = append(args, checkType)
}
return requestCheckScript.Run(ctx, s.redis,
[]string{constants.RedisCardDueKey(request.CardID), constants.RedisCarrierPriorityQueueKey(request.CarrierID),
constants.RedisCarrierQueueKey(request.CarrierID), constants.RedisCardClaimKey(request.CardID)},
args...).Int64()
}
// Remove 把卡移出所属账户队列并删除到期信息(关闭轮询、删除卡)。
func (s *Store) Remove(ctx context.Context, cardID, carrierID uint) error {
return removeScript.Run(ctx, s.redis,
[]string{constants.RedisCarrierPriorityQueueKey(carrierID), constants.RedisCarrierQueueKey(carrierID), constants.RedisCardDueKey(cardID)},
cardID).Err()
}
// Upsert 写入卡各检查类型到期时间并按最早到期放入账户队列(已在优先队列中的保留在优先队列)。
// 卡所属账户变化时先移出旧账户队列。dues 为空表示该卡没有任何需要执行的检查类型,移出队列。
func (s *Store) Upsert(ctx context.Context, cardID, carrierID uint, dues map[string]time.Time) error {
dueKey := constants.RedisCardDueKey(cardID)
previous, err := s.redis.HGet(ctx, dueKey, fieldCarrier).Uint64()
if err != nil && !stderrors.Is(err, redis.Nil) {
return err
}
pipe := s.redis.TxPipeline()
if previous != 0 && uint(previous) != carrierID {
pipe.ZRem(ctx, constants.RedisCarrierPriorityQueueKey(uint(previous)), cardID)
pipe.ZRem(ctx, constants.RedisCarrierQueueKey(uint(previous)), cardID)
}
if len(dues) == 0 {
pipe.ZRem(ctx, constants.RedisCarrierPriorityQueueKey(carrierID), cardID)
pipe.ZRem(ctx, constants.RedisCarrierQueueKey(carrierID), cardID)
pipe.Del(ctx, dueKey)
_, err := pipe.Exec(ctx)
return err
}
values := map[string]any{fieldCarrier: carrierID}
var earliest time.Time
for _, checkType := range constants.AllCheckTypes() {
due, ok := dues[checkType]
if !ok {
pipe.HDel(ctx, dueKey, fieldDue+checkType)
continue
}
values[fieldDue+checkType] = due.UnixMilli()
if earliest.IsZero() || due.Before(earliest) {
earliest = due
}
}
pipe.HSet(ctx, dueKey, values)
_, err = pipe.Exec(ctx)
if err != nil {
return err
}
priorityKey := constants.RedisCarrierPriorityQueueKey(carrierID)
if inPriority, _ := s.redis.ZScore(ctx, priorityKey, strconv.FormatUint(uint64(cardID), 10)).Result(); inPriority > 0 {
return s.redis.ZAdd(ctx, priorityKey, redis.Z{Score: float64(earliest.UnixMilli()), Member: cardID}).Err()
}
return s.redis.ZAdd(ctx, constants.RedisCarrierQueueKey(carrierID), redis.Z{Score: float64(earliest.UnixMilli()), Member: cardID}).Err()
}
// CardDue 是卡到期信息的解析结果。
type CardDue struct {
CarrierID uint
Priority bool
Types map[string]TypeDue
}
// TypeDue 是卡某检查类型的到期与失败信息。
type TypeDue struct {
Due time.Time
HasDue bool
LastOK time.Time
FailCount int
FirstFailed time.Time
LastError string
Confirm []time.Duration
Expected string
}
// ReadDue 读取并解析卡到期信息;Hash 不存在时返回空结构。
func (s *Store) ReadDue(ctx context.Context, cardID uint) (CardDue, error) {
raw, err := s.redis.HGetAll(ctx, constants.RedisCardDueKey(cardID)).Result()
if err != nil {
return CardDue{}, err
}
return ParseDue(raw), nil
}
// ParseDue 解析卡到期 Hash。
func ParseDue(raw map[string]string) CardDue {
result := CardDue{Types: map[string]TypeDue{}}
if value, err := strconv.ParseUint(raw[fieldCarrier], 10, 64); err == nil {
result.CarrierID = uint(value)
}
result.Priority = raw[fieldPriority] == "1"
for _, checkType := range constants.AllCheckTypes() {
item := TypeDue{}
if ms, err := strconv.ParseInt(raw[fieldDue+checkType], 10, 64); err == nil {
item.Due, item.HasDue = time.UnixMilli(ms), true
}
if ms, err := strconv.ParseInt(raw[fieldOK+checkType], 10, 64); err == nil && ms > 0 {
item.LastOK = time.UnixMilli(ms)
}
item.FailCount, _ = strconv.Atoi(raw[fieldFail+checkType])
if ms, err := strconv.ParseInt(raw[fieldFirstFail+checkType], 10, 64); err == nil && ms > 0 {
item.FirstFailed = time.UnixMilli(ms)
}
item.LastError = raw[fieldError+checkType]
for _, part := range strings.Split(raw[fieldConfirm+checkType], ",") {
if seconds, err := strconv.Atoi(strings.TrimSpace(part)); err == nil && seconds > 0 {
item.Confirm = append(item.Confirm, time.Duration(seconds)*time.Second)
}
}
item.Expected = raw[fieldExpected+checkType]
result.Types[checkType] = item
}
return result
}
// Field 名称导出给执行器组装收尾写入。
func FieldDue(checkType string) string { return fieldDue + checkType }
func FieldOK(checkType string) string { return fieldOK + checkType }
func FieldFail(checkType string) string { return fieldFail + checkType }
func FieldFirstFail(checkType string) string { return fieldFirstFail + checkType }
func FieldError(checkType string) string { return fieldError + checkType }
func FieldConfirm(checkType string) string { return fieldConfirm + checkType }
func FieldExpected(checkType string) string { return fieldExpected + checkType }

View File

@@ -0,0 +1,161 @@
package polling
import (
"context"
"encoding/json"
"strings"
"sync"
"time"
"go.uber.org/zap"
"gorm.io/gorm"
"github.com/break/junhong_cmp_fiber/internal/model"
"github.com/break/junhong_cmp_fiber/internal/polling/carrierqueue"
"github.com/break/junhong_cmp_fiber/pkg/constants"
)
// sceneCacheTTL 是请求检查场景配置的进程内缓存时长。
const sceneCacheTTL = 30 * time.Second
// SceneConfig 是一个请求检查场景的生效配置。
type SceneConfig struct {
Scene string
CheckTypes []string
ConfirmDelays []time.Duration
Cooldown time.Duration
}
// CheckRequester 按场景配置为卡请求检查(业务事件与读入口):把对应检查类型设为立即到期放入账户优先队列,
// 同卡多次请求合并、冷却期内不请求、按场景追加延迟确认。只在按运营商账户调度时生效。
type CheckRequester struct {
scheduling *CarrierScheduling
db *gorm.DB
logger *zap.Logger
mu sync.Mutex
scenes map[string]SceneConfig
loadedAt time.Time
}
// NewCheckRequester 创建请求检查器。
func NewCheckRequester(scheduling *CarrierScheduling, db *gorm.DB, logger *zap.Logger) *CheckRequester {
if logger == nil {
logger = zap.NewNop()
}
return &CheckRequester{scheduling: scheduling, db: db, logger: logger}
}
// Active 判断当前是否按运营商账户调度(请求检查生效)。
func (r *CheckRequester) Active(ctx context.Context) bool {
return r != nil && r.scheduling.Active(ctx)
}
// RequestCards 为一组卡按场景请求检查;expected 为延迟确认的期望结果(online / offline,可空)。
// 失败只记日志,不影响业务。
func (r *CheckRequester) RequestCards(ctx context.Context, scene string, cardIDs []uint, expected string) {
if r == nil || len(cardIDs) == 0 {
return
}
config, ok := r.sceneConfig(ctx, scene)
if !ok || len(config.CheckTypes) == 0 {
return
}
carriers, err := r.cardCarriers(ctx, cardIDs)
if err != nil {
r.logger.Warn("请求检查:查询卡所属运营商账户失败", zap.String("scene", scene), zap.Error(err))
return
}
store := r.scheduling.Store()
for _, cardID := range cardIDs {
carrierID, found := carriers[cardID]
if !found {
continue
}
if _, err := store.RequestCheck(ctx, carrierqueue.CheckRequest{
CardID: cardID, CarrierID: carrierID, CheckTypes: config.CheckTypes, Cooldown: config.Cooldown,
ConfirmDelays: config.ConfirmDelays, Expected: expected,
}); err != nil {
r.logger.Warn("请求检查写入账户队列失败", zap.Uint("card_id", cardID), zap.String("scene", scene), zap.Error(err))
}
}
}
// RequestDevice 为设备下全部有效绑定卡按场景请求检查。
func (r *CheckRequester) RequestDevice(ctx context.Context, scene string, deviceID uint) {
if r == nil || deviceID == 0 {
return
}
var cardIDs []uint
if err := r.db.WithContext(ctx).Model(&model.DeviceSimBinding{}).
Where("device_id = ? AND bind_status = ?", deviceID, constants.BindStatusBound).Pluck("iot_card_id", &cardIDs).Error; err != nil {
r.logger.Warn("请求检查:查询设备绑定卡失败", zap.Uint("device_id", deviceID), zap.Error(err))
return
}
r.RequestCards(ctx, scene, cardIDs, "")
}
func (r *CheckRequester) cardCarriers(ctx context.Context, cardIDs []uint) (map[uint]uint, error) {
var cards []model.IotCard
if err := r.db.WithContext(ctx).Select("id", "carrier_id").Where("id IN ?", cardIDs).Find(&cards).Error; err != nil {
return nil, err
}
result := make(map[uint]uint, len(cards))
for _, card := range cards {
result[card.ID] = card.CarrierID
}
return result, nil
}
// sceneConfig 读取场景配置(缓存);场景未预置时不请求。
func (r *CheckRequester) sceneConfig(ctx context.Context, scene string) (SceneConfig, bool) {
r.mu.Lock()
defer r.mu.Unlock()
if r.scenes == nil || time.Since(r.loadedAt) >= sceneCacheTTL {
var rows []model.PollingCheckScene
if err := r.db.WithContext(ctx).Find(&rows).Error; err != nil {
r.logger.Warn("读取请求检查场景失败,沿用上一次配置", zap.Error(err))
} else {
scenes := make(map[string]SceneConfig, len(rows))
for _, row := range rows {
scenes[row.Scene] = ParseSceneConfig(row)
}
r.scenes, r.loadedAt = scenes, time.Now()
}
}
config, ok := r.scenes[scene]
return config, ok
}
// ParseSceneConfig 解析场景配置行。
func ParseSceneConfig(row model.PollingCheckScene) SceneConfig {
config := SceneConfig{Scene: row.Scene}
var types []string
_ = json.Unmarshal(row.CheckTypes, &types)
for _, checkType := range types {
if constants.CheckTypeCallsGateway(strings.TrimSpace(checkType)) {
config.CheckTypes = append(config.CheckTypes, strings.TrimSpace(checkType))
}
}
var delays []int
_ = json.Unmarshal(row.ConfirmDelays, &delays)
for _, seconds := range delays {
if seconds > 0 {
config.ConfirmDelays = append(config.ConfirmDelays, time.Duration(seconds)*time.Second)
}
}
if row.Cooldown != nil && *row.Cooldown > 0 {
config.Cooldown = time.Duration(*row.Cooldown) * time.Second
}
return config
}
// InvalidateScenes 使场景缓存立即失效(后台修改后调用)。
func (r *CheckRequester) InvalidateScenes() {
if r == nil {
return
}
r.mu.Lock()
r.loadedAt = time.Time{}
r.mu.Unlock()
}

View File

@@ -25,12 +25,24 @@ type PollingConfigManager struct {
mu sync.RWMutex
configs []*model.PollingConfig
refreshOnce sync.Once
tier *TierResolver
}
// SetTierResolver 注入分层事实解析器;未注入时要求套餐状态的配置不匹配。
func (m *PollingConfigManager) SetTierResolver(tier *TierResolver) {
m.tier = tier
}
// TierResolver 返回已注入的分层事实解析器。
func (m *PollingConfigManager) TierResolver() *TierResolver {
return m.tier
}
// TaskTypeInterval 某任务类型的轮询间隔信息
type TaskTypeInterval struct {
Interval int // 秒
Priority int // 来源配置的 priority
Interval int // 秒
Priority int // 来源配置的 priority
ConfigID uint // 来源配置 ID
}
// NewPollingConfigManager 创建配置管理器
@@ -149,74 +161,153 @@ func (m *PollingConfigManager) MatchConfig(card *model.IotCard) *model.PollingCo
return configs[0]
}
// MatchConfigs 按 priority ASC 遍历所有启用的配置,返回所有满足匹配条件且至少有一个非 NULL interval 的配置
// 无匹配时返回空切片
// MatchConfigs 按 priority、id 升序遍历所有启用的配置,返回满足匹配条件且至少配置了一种检查间隔(含 0 不检查)的配置。
// 无匹配时返回空切片。
func (m *PollingConfigManager) MatchConfigs(card *model.IotCard) []*model.PollingConfig {
m.mu.RLock()
defer m.mu.RUnlock()
configs := m.configs
m.mu.RUnlock()
return m.matchAgainst(configs, card)
}
func (m *PollingConfigManager) matchAgainst(configs []*model.PollingConfig, card *model.IotCard) []*model.PollingConfig {
facts := m.lazyTierFacts(card)
var result []*model.PollingConfig
for _, cfg := range m.configs {
if matchConfigConditions(cfg, card) && hasAnyEnabledInterval(cfg) {
for _, cfg := range configs {
if matchConfigConditions(cfg, card, facts) && hasAnyConfiguredInterval(cfg) {
result = append(result, cfg)
}
}
return result
}
// hasAnyEnabledInterval 检查配置是否有至少一个非 NULL 且大于 0 的轮询间隔
func hasAnyEnabledInterval(cfg *model.PollingConfig) bool {
return (cfg.RealnameCheckInterval != nil && *cfg.RealnameCheckInterval > 0) ||
(cfg.CarddataCheckInterval != nil && *cfg.CarddataCheckInterval > 0) ||
(cfg.PackageCheckInterval != nil && *cfg.PackageCheckInterval > 0) ||
(cfg.ProtectCheckInterval != nil && *cfg.ProtectCheckInterval > 0) ||
(cfg.CardStatusCheckInterval != nil && *cfg.CardStatusCheckInterval > 0)
// lazyTierFacts 只在某个配置要求套餐状态时才解析卡的分层事实,避免无分层配置时额外查库。
func (m *PollingConfigManager) lazyTierFacts(card *model.IotCard) func() TierFacts {
var once sync.Once
var facts TierFacts
return func() TierFacts {
once.Do(func() {
if m.tier != nil {
ctx, cancel := context.WithTimeout(context.Background(), 3*time.Second)
defer cancel()
facts = m.tier.Resolve(ctx, card)
}
})
return facts
}
}
// MergedTaskIntervals 对所有匹配配置按 priority 遍历,对每种 task type 选取最高优先级的非 nil interval
// hasAnyConfiguredInterval 检查配置是否至少配置了一种检查间隔(正数或 0)。
func hasAnyConfiguredInterval(cfg *model.PollingConfig) bool {
return cfg.RealnameCheckInterval != nil || cfg.CarddataCheckInterval != nil || cfg.PackageCheckInterval != nil ||
cfg.CardStatusCheckInterval != nil
}
// IntervalSource 是某检查类型最终采用的间隔及其来源,供合并结果与匹配预览共用。
type IntervalSource struct {
TaskType string
Interval int // 秒;Disabled 为 true 时为 0
Disabled bool // 命中取值 0(不检查)
ConfigID uint // 来源配置;未匹配时为 0
Priority int
Config *model.PollingConfig
}
// taskIntervalFields 返回配置中各检查类型的间隔字段。
func taskIntervalFields(cfg *model.PollingConfig) []struct {
taskType string
value *int
} {
return []struct {
taskType string
value *int
}{
{constants.TaskTypePollingRealname, cfg.RealnameCheckInterval},
{constants.TaskTypePollingCarddata, cfg.CarddataCheckInterval},
{constants.TaskTypePollingPackage, cfg.PackageCheckInterval},
{constants.TaskTypePollingCardStatus, cfg.CardStatusCheckInterval},
}
}
// ResolveIntervalSources 按三态规则合并匹配配置:每种检查类型取第一个非空取值,正数按间隔、0 不检查且停止向下查找、空继承。
func ResolveIntervalSources(configs []*model.PollingConfig) map[string]IntervalSource {
result := make(map[string]IntervalSource)
for _, cfg := range configs {
for _, field := range taskIntervalFields(cfg) {
if field.value == nil {
continue
}
if _, exists := result[field.taskType]; exists {
continue
}
source := IntervalSource{TaskType: field.taskType, ConfigID: cfg.ID, Priority: cfg.Priority, Config: cfg}
if *field.value <= 0 {
source.Disabled = true
} else {
source.Interval = *field.value
}
result[field.taskType] = source
}
}
return result
}
// MergedTaskIntervals 返回卡各检查类型最终采用的正数间隔;取值 0(不检查)的类型不出现在结果中。
// 返回 map: taskType → TaskTypeInterval{Interval, Priority}
func (m *PollingConfigManager) MergedTaskIntervals(card *model.IotCard) map[string]TaskTypeInterval {
configs := m.MatchConfigs(card)
merged := make(map[string]TaskTypeInterval)
for _, cfg := range configs {
priority := int(cfg.Priority)
if cfg.RealnameCheckInterval != nil && *cfg.RealnameCheckInterval > 0 {
taskType := constants.TaskTypePollingRealname
if _, exists := merged[taskType]; !exists {
merged[taskType] = TaskTypeInterval{Interval: *cfg.RealnameCheckInterval, Priority: priority}
}
}
if cfg.CarddataCheckInterval != nil && *cfg.CarddataCheckInterval > 0 {
taskType := constants.TaskTypePollingCarddata
if _, exists := merged[taskType]; !exists {
merged[taskType] = TaskTypeInterval{Interval: *cfg.CarddataCheckInterval, Priority: priority}
}
}
if cfg.PackageCheckInterval != nil && *cfg.PackageCheckInterval > 0 {
taskType := constants.TaskTypePollingPackage
if _, exists := merged[taskType]; !exists {
merged[taskType] = TaskTypeInterval{Interval: *cfg.PackageCheckInterval, Priority: priority}
}
}
if cfg.ProtectCheckInterval != nil && *cfg.ProtectCheckInterval > 0 {
taskType := constants.TaskTypePollingProtect
if _, exists := merged[taskType]; !exists {
merged[taskType] = TaskTypeInterval{Interval: *cfg.ProtectCheckInterval, Priority: priority}
}
}
if cfg.CardStatusCheckInterval != nil && *cfg.CardStatusCheckInterval > 0 {
taskType := constants.TaskTypePollingCardStatus
if _, exists := merged[taskType]; !exists {
merged[taskType] = TaskTypeInterval{Interval: *cfg.CardStatusCheckInterval, Priority: priority}
}
sources := ResolveIntervalSources(m.MatchConfigs(card))
merged := make(map[string]TaskTypeInterval, len(sources))
for taskType, source := range sources {
if source.Disabled {
continue
}
merged[taskType] = TaskTypeInterval{Interval: source.Interval, Priority: source.Priority, ConfigID: source.ConfigID}
}
if len(sources) == 0 {
m.warnUnmatchedFallback(card)
}
return merged
}
// matchConfigConditions 检查卡是否满足配置的匹配条件
func matchConfigConditions(cfg *model.PollingConfig, card *model.IotCard) bool {
// PreviewIntervalSources 用给定配置集合(可含未保存修改)计算卡的匹配配置与最终间隔,只读。
func (m *PollingConfigManager) PreviewIntervalSources(configs []*model.PollingConfig, card *model.IotCard) ([]*model.PollingConfig, map[string]IntervalSource) {
matched := m.matchAgainst(configs, card)
return matched, ResolveIntervalSources(matched)
}
// EnabledConfigs 返回当前内存中的启用配置快照(已按 priority、id 排序)。
func (m *PollingConfigManager) EnabledConfigs() []*model.PollingConfig {
m.mu.RLock()
defer m.mu.RUnlock()
out := make([]*model.PollingConfig, len(m.configs))
copy(out, m.configs)
return out
}
// TierFactsOf 返回卡的分层事实(未配置解析器时 Known 为 false)。
func (m *PollingConfigManager) TierFactsOf(card *model.IotCard) TierFacts {
return m.lazyTierFacts(card)()
}
// warnUnmatchedFallback 已销售无套餐、在库的卡未匹配任何配置时告警:这两类卡依赖卡状态检查兜底开机即停。
func (m *PollingConfigManager) warnUnmatchedFallback(card *model.IotCard) {
if card == nil || m.logger == nil {
return
}
if card.AssetStatus == constants.AssetStatusInStock {
m.logger.Warn("在库卡未匹配任何轮询配置,将不入队", zap.Uint("card_id", card.ID), zap.String("metric", "polling.tier.unmatched_in_stock"))
return
}
if card.AssetStatus == constants.AssetStatusSold {
facts := m.TierFactsOf(card)
if facts.Known && facts.PackageState == constants.PollingPackageStateNone {
m.logger.Warn("已销售无套餐卡未匹配任何轮询配置,将不入队", zap.Uint("card_id", card.ID), zap.String("metric", "polling.tier.unmatched_sold_no_package"))
}
}
}
// matchConfigConditions 检查卡是否满足配置的匹配条件;套餐状态仅在配置要求时解析,解析失败视为不匹配。
func matchConfigConditions(cfg *model.PollingConfig, card *model.IotCard, facts func() TierFacts) bool {
if cfg.CardCondition != "" {
if cfg.CardCondition != getCardCondition(card) {
return false
@@ -232,9 +323,23 @@ func matchConfigConditions(cfg *model.PollingConfig, card *model.IotCard) bool {
return false
}
}
if cfg.AssetStatus != nil && *cfg.AssetStatus != card.AssetStatus {
return false
}
if cfg.PackageState != nil && *cfg.PackageState != "" {
f := facts()
if !f.Known || f.PackageState != *cfg.PackageState {
return false
}
}
return true
}
// CardConditionOf 返回卡用于匹配轮询配置的状态条件(not_real_name / activated / suspended)。
func CardConditionOf(card *model.IotCard) string {
return getCardCondition(card)
}
// getCardCondition 获取卡的状态条件(用于匹配轮询配置)
// ⚠️ 注意判断顺序:停机优先,避免停机卡错误匹配 activated 或 not_real_name 配置
// 停机卡需要继续轮询 carddata/package 以检测复机条件(套餐购买、流量重置、实名完成)

View File

@@ -257,6 +257,10 @@ func (p *PollingInitializer) initBatch(ctx context.Context, cards []*model.IotCa
cmdCount = 0
}
// 分层轮询按批预取套餐状态,避免逐卡查库。
if tier := p.configMgr.TierResolver(); tier != nil {
tier.Prefill(ctx, cards)
}
for _, card := range cards {
intervals := p.configMgr.MergedTaskIntervals(card)
if len(intervals) == 0 {
@@ -271,6 +275,10 @@ func (p *PollingInitializer) initBatch(ctx context.Context, cards []*model.IotCa
for taskType, info := range intervals {
lastCheckAt := lastCheckAtByTaskType(card, taskType)
nextCheck := calculateNextCheckTime(lastCheckAt, info.Interval, now)
// 重置日前加密:窗口规则算出的时间更早时(如需截断到窗口起点)以它为准。
if windowDue := p.configMgr.NextCheckAt(ctx, card, taskType, info, now); windowDue.Before(nextCheck) && windowDue.After(now) {
nextCheck = windowDue
}
pipe.ZAdd(ctx, constants.RedisPollingShardQueueKey(shardID, taskType), redis.Z{
Score: float64(nextCheck.Unix()), Member: cardIDStr,
})
@@ -289,7 +297,8 @@ func (p *PollingInitializer) initBatch(ctx context.Context, cards []*model.IotCa
"last_gateway_reading_mb": card.LastGatewayReadingMB,
"data_usage_mb": card.DataUsageMB,
"stop_reason": card.StopReason, "is_standalone": boolToStr(card.IsStandalone),
"cached_at": now.Unix(),
"asset_status": card.AssetStatus,
"cached_at": now.Unix(),
}
if card.CurrentMonthStartDate != nil {
cacheData["current_month_start_date"] = card.CurrentMonthStartDate.Unix()
@@ -318,8 +327,6 @@ func lastCheckAtByTaskType(card *model.IotCard, taskType string) *time.Time {
return card.LastRealNameCheckAt
case constants.TaskTypePollingCarddata, constants.TaskTypePollingPackage:
return card.LastDataCheckAt
case constants.TaskTypePollingProtect:
return card.LastProtectCheckAt
case constants.TaskTypePollingCardStatus:
return card.LastCardStatusCheckAt
default:

View File

@@ -22,6 +22,18 @@ type PollingLifecycleService struct {
deviceBindingStore *postgres.DeviceSimBindingStore
deviceStore *postgres.DeviceStore
logger *zap.Logger
// carrier 按运营商账户调度接缝;当前模式为 carrier 时生命周期事件写入账户队列,不写旧分片队列。
carrier *CarrierScheduling
}
// SetCarrierScheduling 注入按运营商账户调度接缝。
func (s *PollingLifecycleService) SetCarrierScheduling(carrier *CarrierScheduling) {
s.carrier = carrier
}
// carrierActive 判断当前是否按运营商账户调度。
func (s *PollingLifecycleService) carrierActive(ctx context.Context) bool {
return s.carrier.Active(ctx)
}
// NewPollingLifecycleService 创建卡生命周期轮询管理服务
@@ -52,6 +64,10 @@ func (s *PollingLifecycleService) OnCardCreated(ctx context.Context, card *model
s.logger.Debug("卡禁用轮询,跳过入队", zap.Uint("card_id", card.ID))
return
}
if s.carrierActive(ctx) {
s.carrier.UpsertCard(ctx, card, false)
return
}
s.enqueueCard(ctx, card)
}
@@ -64,6 +80,11 @@ func (s *PollingLifecycleService) OnBatchCardsCreated(ctx context.Context, cards
// OnCardStatusChanged 卡状态变化后清理缓存、重新匹配配置并更新队列
func (s *PollingLifecycleService) OnCardStatusChanged(ctx context.Context, cardID uint) {
if s.carrierActive(ctx) {
s.queueMgr.InvalidateCardCache(ctx, cardID)
s.carrierRefresh(ctx, cardID)
return
}
if err := s.queueMgr.RemoveFromAllQueues(ctx, cardID); err != nil {
s.logger.Warn("卡状态变化:从队列移除失败", zap.Uint("card_id", cardID), zap.Error(err))
}
@@ -87,6 +108,13 @@ func (s *PollingLifecycleService) OnBatchCardsStatusChanged(ctx context.Context,
if len(cardIDs) == 0 {
return
}
if s.carrierActive(ctx) {
for _, cardID := range cardIDs {
s.queueMgr.InvalidateCardCache(ctx, cardID)
s.carrierRefresh(ctx, cardID)
}
return
}
// 单次 pipeline:批量从当前分片队列移除并清理缓存
if err := s.queueMgr.RemoveBatchFromCurrentShardQueues(ctx, cardIDs); err != nil {
s.logger.Warn("批量卡状态变化:从队列移除失败", zap.Int("count", len(cardIDs)), zap.Error(err))
@@ -107,6 +135,11 @@ func (s *PollingLifecycleService) OnBatchCardsStatusChanged(ctx context.Context,
// OnCardDeleted 卡删除后移除所有队列并清理缓存
func (s *PollingLifecycleService) OnCardDeleted(ctx context.Context, cardID uint) {
if s.carrierActive(ctx) {
s.carrier.RemoveCard(ctx, cardID, 0)
s.queueMgr.InvalidateCardCache(ctx, cardID)
return
}
if err := s.queueMgr.OnCardDeleted(ctx, cardID); err != nil {
s.logger.Warn("卡删除:清理队列和缓存失败", zap.Uint("card_id", cardID), zap.Error(err))
}
@@ -123,11 +156,21 @@ func (s *PollingLifecycleService) OnCardEnabled(ctx context.Context, cardID uint
if !s.ShouldEnqueue(ctx, card) {
return
}
if s.carrierActive(ctx) {
// 恢复轮询时各检查类型到期设为当前时间加抖动。
s.carrier.UpsertCard(ctx, card, false)
return
}
s.enqueueCard(ctx, card)
}
// OnCardDisabled 卡禁用轮询后移除所有队列
func (s *PollingLifecycleService) OnCardDisabled(ctx context.Context, cardID uint) {
if s.carrierActive(ctx) {
s.carrier.RemoveCard(ctx, cardID, 0)
s.queueMgr.InvalidateCardCache(ctx, cardID)
return
}
if err := s.queueMgr.RemoveFromAllQueues(ctx, cardID); err != nil {
s.logger.Warn("卡禁用:从队列移除失败", zap.Uint("card_id", cardID), zap.Error(err))
}
@@ -183,3 +226,17 @@ func calcInitialDelay(interval int) time.Time {
}
return now.Add(time.Duration(rand.Intn(jitterMax)) * time.Second)
}
// carrierRefresh 按运营商账户调度时重新匹配配置并写入账户队列:不再轮询的卡移出,其余按当前时间加抖动重排。
func (s *PollingLifecycleService) carrierRefresh(ctx context.Context, cardID uint) {
card, err := s.iotCardStore.GetByID(ctx, cardID)
if err != nil {
s.logger.Error("卡状态变化:加载卡信息失败", zap.Uint("card_id", cardID), zap.Error(err))
return
}
if !s.ShouldEnqueue(ctx, card) {
s.carrier.RemoveCard(ctx, card.ID, card.CarrierID)
return
}
s.carrier.UpsertCard(ctx, card, false)
}

View File

@@ -0,0 +1,134 @@
package polling
import (
"context"
"strconv"
"time"
"github.com/break/junhong_cmp_fiber/internal/model"
"github.com/break/junhong_cmp_fiber/pkg/constants"
)
// shanghaiLocation 返回业务时区(上海)。
func shanghaiLocation() *time.Location {
location, err := time.LoadLocation("Asia/Shanghai")
if err != nil {
return time.FixedZone("CST", 8*3600)
}
return location
}
// nextResetAt 返回 now 之后最近一次流量重置时刻:重置日当天 0 点(上海时区)。
func nextResetAt(now time.Time, resetDay int) time.Time {
if resetDay < 1 || resetDay > 28 {
resetDay = 1
}
local := now.In(shanghaiLocation())
candidate := time.Date(local.Year(), local.Month(), resetDay, 0, 0, 0, 0, shanghaiLocation())
if !candidate.After(local) {
candidate = candidate.AddDate(0, 1, 0)
}
return candidate
}
// PreResetNextDue 计算重置日前加密后的流量检查下次到期时间。
// - 当前在窗口 [reset-window, reset) 内:下次 = now + 窗口间隔;越过重置时刻则改为 reset - 收尾提前量,已过则按正常间隔;
// - 当前在窗口前、按正常间隔算出的下次越过窗口起点:截断到窗口起点,保证按时进入窗口;
// - 其余按正常间隔。
func PreResetNextDue(now time.Time, normalInterval, window, windowInterval time.Duration, reset time.Time, tail time.Duration) time.Time {
normal := now.Add(normalInterval)
windowStart := reset.Add(-window)
if !now.Before(windowStart) && now.Before(reset) {
next := now.Add(windowInterval)
if next.Before(reset) {
return next
}
last := reset.Add(-tail)
if last.After(now) {
return last
}
return normal
}
if now.Before(windowStart) && normal.After(windowStart) {
return windowStart
}
return normal
}
// NextCheckAt 返回卡某检查类型的下次到期时间:流量检查在来源配置开启重置日前加密时按窗口规则计算,其余为 now + 间隔。
func (m *PollingConfigManager) NextCheckAt(ctx context.Context, card *model.IotCard, taskType string, info TaskTypeInterval, now time.Time) time.Time {
normal := now.Add(time.Duration(info.Interval) * time.Second)
if taskType != constants.TaskTypePollingCarddata || card == nil || info.ConfigID == 0 {
return normal
}
cfg := m.configByID(info.ConfigID)
if cfg == nil || cfg.PreResetWindow == nil || cfg.PreResetCarddataInterval == nil {
return normal
}
resetDay := 1
if m.tier != nil {
resetDay = m.tier.ResetDay(ctx, card.CarrierID)
}
return PreResetNextDue(now, time.Duration(info.Interval)*time.Second,
time.Duration(*cfg.PreResetWindow)*time.Second, time.Duration(*cfg.PreResetCarddataInterval)*time.Second,
nextResetAt(now, resetDay), constants.PollingPreResetTailMinutes*time.Minute)
}
func (m *PollingConfigManager) configByID(id uint) *model.PollingConfig {
m.mu.RLock()
defer m.mu.RUnlock()
for _, cfg := range m.configs {
if cfg.ID == id {
return cfg
}
}
return nil
}
// RecordPreResetCheck 在流量查询成功后记录重置日前加密效果:处于该卡来源配置的加密窗口内时,
// 把卡记入窗口已查集合,并记下本次查询距重置时刻的分钟数(同卡取最后一次)。统计失败不影响轮询。
func (m *PollingConfigManager) RecordPreResetCheck(ctx context.Context, card *model.IotCard, now time.Time) {
if m == nil || m.redis == nil || card == nil {
return
}
info, ok := m.MergedTaskIntervals(card)[constants.TaskTypePollingCarddata]
if !ok {
return
}
cfg := m.configByID(info.ConfigID)
if cfg == nil || cfg.PreResetWindow == nil || cfg.PreResetCarddataInterval == nil {
return
}
resetDay := 1
if m.tier != nil {
resetDay = m.tier.ResetDay(ctx, card.CarrierID)
}
reset := nextResetAt(now, resetDay)
if now.Before(reset.Add(-time.Duration(*cfg.PreResetWindow) * time.Second)) {
return
}
// 只统计有生效套餐的卡,与统计接口"有生效套餐但窗口内未查到"的口径一致。
if facts := m.TierFactsOf(card); !facts.Known || facts.PackageState != constants.PollingPackageStateActive {
return
}
date := reset.Format("20060102")
member := strconv.FormatUint(uint64(card.ID), 10)
minutes := int(reset.Sub(now).Minutes())
ttl := constants.PollingPreResetStatTTLSeconds * time.Second
pipe := m.redis.Pipeline()
pipe.PFAdd(ctx, constants.RedisPollingPreResetCheckedKey(card.CarrierID, date), member)
pipe.Expire(ctx, constants.RedisPollingPreResetCheckedKey(card.CarrierID, date), ttl)
pipe.HSet(ctx, constants.RedisPollingPreResetOffsetKey(card.CarrierID, date), member, minutes)
pipe.Expire(ctx, constants.RedisPollingPreResetOffsetKey(card.CarrierID, date), ttl)
_, _ = pipe.Exec(ctx)
}
// ResetMoments 返回某运营商账户最近一次已过去的重置时刻与下一次重置时刻。
func (m *PollingConfigManager) ResetMoments(ctx context.Context, carrierID uint, now time.Time) (time.Time, time.Time) {
resetDay := 1
if m.tier != nil {
resetDay = m.tier.ResetDay(ctx, carrierID)
}
next := nextResetAt(now, resetDay)
return next.AddDate(0, -1, 0), next
}

View File

@@ -17,7 +17,6 @@ var allTaskTypes = []string{
constants.TaskTypePollingRealname,
constants.TaskTypePollingCarddata,
constants.TaskTypePollingPackage,
constants.TaskTypePollingProtect,
constants.TaskTypePollingCardStatus,
}

View File

@@ -0,0 +1,179 @@
package polling
import (
"context"
"fmt"
"strconv"
"time"
"github.com/redis/go-redis/v9"
"go.uber.org/zap"
"gorm.io/gorm"
"github.com/break/junhong_cmp_fiber/internal/model"
"github.com/break/junhong_cmp_fiber/pkg/constants"
)
// recalculateBatchSize 是重算到期时间每批读取的卡数。
const recalculateBatchSize = 500
// RecalculateProgress 是重算到期时间任务的进度快照。
type RecalculateProgress struct {
Total int
Processed int
Advanced int // 被提前的检查项数
}
// DueRecalculator 按当前轮询配置重算已在队列中的卡的下次到期时间:新到期 = min(当前到期, 上次检查 + 新间隔),只提前不推后。
// 只更新已在分片队列中的成员(ZADD XX LT),不新增成员,避免把执行中的卡重复放回队列;不调用网关。
type DueRecalculator struct {
db *gorm.DB
redis *redis.Client
configMgr *PollingConfigManager
queueMgr *PollingQueueManager
logger *zap.Logger
// carrier 按运营商账户调度接缝:carrier 模式下重算写入卡到期信息与账户队列。
carrier *CarrierScheduling
}
// SetCarrierScheduling 注入按运营商账户调度接缝。
func (r *DueRecalculator) SetCarrierScheduling(carrier *CarrierScheduling) {
r.carrier = carrier
}
// recalculateCarrierBatch 按运营商账户调度时重算:各检查类型候选到期同旧结构口径,只提前不推后。
func (r *DueRecalculator) recalculateCarrierBatch(ctx context.Context, cards []*model.IotCard) (int, error) {
now := time.Now()
advanced := 0
for _, card := range cards {
for taskType, info := range r.configMgr.MergedTaskIntervals(card) {
checkType := constants.CheckTypeFromTaskType(taskType)
if checkType == "" || info.Interval <= 0 {
continue
}
due := recalculatedDue(lastCheckAtByTaskType(card, taskType), info.Interval, now)
if windowDue := r.configMgr.NextCheckAt(ctx, card, taskType, info, now); windowDue.Before(due) && windowDue.After(now) {
due = windowDue
}
ok, err := r.carrier.AdvanceDue(ctx, card, checkType, due)
if err != nil {
return advanced, err
}
if ok {
advanced++
}
}
}
return advanced, nil
}
// NewDueRecalculator 创建重算器。
func NewDueRecalculator(db *gorm.DB, redisClient *redis.Client, configMgr *PollingConfigManager, queueMgr *PollingQueueManager, logger *zap.Logger) *DueRecalculator {
if logger == nil {
logger = zap.NewNop()
}
return &DueRecalculator{db: db, redis: redisClient, configMgr: configMgr, queueMgr: queueMgr, logger: logger}
}
// Run 执行重算;carrierID 非空时只处理该运营商账户的卡。每处理一批回调一次进度。
func (r *DueRecalculator) Run(ctx context.Context, carrierID *uint, report func(RecalculateProgress)) (RecalculateProgress, error) {
var progress RecalculateProgress
if r == nil || r.db == nil || r.redis == nil || r.configMgr == nil || r.queueMgr == nil {
return progress, fmt.Errorf("重算到期时间能力未配置")
}
if err := r.configMgr.Load(ctx); err != nil {
return progress, err
}
base := r.db.WithContext(ctx).Model(&model.IotCard{}).Where("enable_polling = true AND deleted_at IS NULL")
if carrierID != nil {
base = base.Where("carrier_id = ?", *carrierID)
}
var total int64
if err := base.Count(&total).Error; err != nil {
return progress, err
}
progress.Total = int(total)
if report != nil {
report(progress)
}
var lastID uint
for {
if err := ctx.Err(); err != nil {
return progress, err
}
var cards []*model.IotCard
query := r.db.WithContext(ctx).Where("id > ? AND enable_polling = true AND deleted_at IS NULL", lastID)
if carrierID != nil {
query = query.Where("carrier_id = ?", *carrierID)
}
if err := query.Order("id ASC").Limit(recalculateBatchSize).Find(&cards).Error; err != nil {
return progress, err
}
if len(cards) == 0 {
break
}
lastID = cards[len(cards)-1].ID
advanced, err := r.recalculateBatch(ctx, cards)
if err != nil {
return progress, err
}
progress.Processed += len(cards)
progress.Advanced += advanced
if report != nil {
report(progress)
}
}
return progress, nil
}
func (r *DueRecalculator) recalculateBatch(ctx context.Context, cards []*model.IotCard) (int, error) {
if tier := r.configMgr.TierResolver(); tier != nil {
tier.Prefill(ctx, cards)
}
if r.carrier.Active(ctx) {
return r.recalculateCarrierBatch(ctx, cards)
}
now := time.Now()
pipe := r.redis.Pipeline()
var cmds []*redis.IntCmd
for _, card := range cards {
shardID := int(card.ID) % r.queueMgr.shardCount
member := strconv.FormatUint(uint64(card.ID), 10)
for taskType, info := range r.configMgr.MergedTaskIntervals(card) {
due := recalculatedDue(lastCheckAtByTaskType(card, taskType), info.Interval, now)
if windowDue := r.configMgr.NextCheckAt(ctx, card, taskType, info, now); windowDue.Before(due) && windowDue.After(now) {
due = windowDue
}
cmds = append(cmds, pipe.ZAddArgs(ctx, constants.RedisPollingShardQueueKey(shardID, taskType), redis.ZAddArgs{
XX: true, LT: true, Ch: true, Members: []redis.Z{{Score: float64(due.Unix()), Member: member}},
}))
}
}
if len(cmds) == 0 {
return 0, nil
}
if _, err := pipe.Exec(ctx); err != nil && err != redis.Nil {
return 0, err
}
advanced := 0
for _, cmd := range cmds {
if n, err := cmd.Result(); err == nil {
advanced += int(n)
}
}
return advanced, nil
}
// recalculatedDue 返回重算的候选到期时间。上次检查时间只在查询成功时更新,查询失败或被限流延后的卡按
// "上次检查 + 新间隔"会算成已过期;此时候选改为"现在 + 新间隔",只提前排队超过一个新间隔的成员,
// 不抵消失败退避与限流延后,也避免重算后集中触发调用。
func recalculatedDue(lastCheckAt *time.Time, intervalSeconds int, now time.Time) time.Time {
interval := time.Duration(intervalSeconds) * time.Second
if lastCheckAt == nil {
return now.Add(interval)
}
if due := lastCheckAt.Add(interval); due.After(now) {
return due
}
return now.Add(interval)
}

View File

@@ -0,0 +1,83 @@
package polling
import (
"context"
"strconv"
"time"
"github.com/bytedance/sonic"
"github.com/hibiken/asynq"
"github.com/redis/go-redis/v9"
"go.uber.org/zap"
"github.com/break/junhong_cmp_fiber/pkg/constants"
)
// RecalculateDuePayload 是重算到期时间异步任务的载荷。
type RecalculateDuePayload struct {
TaskID string `json:"task_id"`
CarrierID *uint `json:"carrier_id,omitempty"`
}
// RecalculateTaskHandler 是重算到期时间的 Asynq 任务处理器;进度写入 Redis 供后台查询。
type RecalculateTaskHandler struct {
recalculator *DueRecalculator
redis *redis.Client
logger *zap.Logger
}
// NewRecalculateTaskHandler 创建重算任务处理器。
func NewRecalculateTaskHandler(recalculator *DueRecalculator, redisClient *redis.Client, logger *zap.Logger) *RecalculateTaskHandler {
if logger == nil {
logger = zap.NewNop()
}
return &RecalculateTaskHandler{recalculator: recalculator, redis: redisClient, logger: logger}
}
// Handle 执行重算;失败写入进度并返回 SkipRetry,避免按指数退避反复全量扫描。
func (h *RecalculateTaskHandler) Handle(ctx context.Context, task *asynq.Task) error {
var payload RecalculateDuePayload
if err := sonic.Unmarshal(task.Payload(), &payload); err != nil || payload.TaskID == "" {
return asynq.SkipRetry
}
key := constants.RedisPollingRecalcProgressKey(payload.TaskID)
h.write(key, map[string]any{"status": constants.PollingRecalcStatusRunning, "started_at": time.Now().Format(time.RFC3339)})
progress, err := h.recalculator.Run(ctx, payload.CarrierID, func(p RecalculateProgress) {
h.write(key, map[string]any{"total": p.Total, "processed": p.Processed, "advanced": p.Advanced})
})
fields := map[string]any{"total": progress.Total, "processed": progress.Processed, "advanced": progress.Advanced, "finished_at": time.Now().Format(time.RFC3339)}
if err != nil {
fields["status"] = constants.PollingRecalcStatusFailed
fields["error"] = "重算到期时间失败,请稍后重试"
h.write(key, fields)
h.logger.Error("重算轮询到期时间失败", zap.String("task_id", payload.TaskID), zap.Error(err))
return asynq.SkipRetry
}
fields["status"] = constants.PollingRecalcStatusCompleted
h.write(key, fields)
h.logger.Info("重算轮询到期时间完成", zap.String("task_id", payload.TaskID),
zap.Int("processed", progress.Processed), zap.Int("advanced", progress.Advanced))
return nil
}
func (h *RecalculateTaskHandler) write(key string, fields map[string]any) {
if h.redis == nil {
return
}
ctx, cancel := context.WithTimeout(context.Background(), 3*time.Second)
defer cancel()
pipe := h.redis.Pipeline()
pipe.HSet(ctx, key, fields)
pipe.Expire(ctx, key, constants.PollingRecalcProgressTTLSeconds*time.Second)
if _, err := pipe.Exec(ctx); err != nil {
h.logger.Warn("写重算到期时间进度失败", zap.String("key", key), zap.Error(err))
}
}
// carrierText 把可选运营商账户转为进度展示文本。
func carrierText(carrierID *uint) string {
if carrierID == nil {
return ""
}
return strconv.FormatUint(uint64(*carrierID), 10)
}

View File

@@ -30,6 +30,8 @@ type Scheduler struct {
packageActivationHandler *PackageActivationHandler
dataResetHandler *DataResetHandler
initializer *PollingInitializer // 可选,nil 表示不守卫 Init 完成
// carrier 按运营商账户调度接缝:当前模式为 carrier 时旧分片调度停止出队(套餐激活检查照常执行)。
carrier *CarrierScheduling
stopChan chan struct{}
wg sync.WaitGroup
@@ -156,6 +158,10 @@ func (s *Scheduler) scheduleLoop(ctx context.Context) {
// processShardSchedule 处理手动队列和分片定时队列(每 1 秒触发)
// 使用 90% 的 tick 间隔作为超时,确保单分片 Redis 挂起时不阻塞下一个 tick
func (s *Scheduler) processShardSchedule(ctx context.Context) {
if s.carrier.Active(ctx) {
// 已切到按运营商账户调度:旧分片调度停止出队,优先提示与手动触发由新分发器消费。
return
}
// 优先提示与手动队列都不受 Init 影响(ConfigManager 已就绪即可)。
// 同一调度周期内先排空优先提示、再排空手动触发队列:优先的强度边界仅为调度入口领先,
// 提示排空不参与分片背压判断,因此分片积压不会抑制加急入队。
@@ -296,3 +302,8 @@ func (s *Scheduler) enqueueBatch(ctx context.Context, taskType string, cardIDs [
}
}
}
// SetCarrierScheduling 注入按运营商账户调度接缝。
func (s *Scheduler) SetCarrierScheduling(carrier *CarrierScheduling) {
s.carrier = carrier
}

168
internal/polling/tier.go Normal file
View File

@@ -0,0 +1,168 @@
package polling
import (
"context"
"sync"
"time"
"go.uber.org/zap"
"github.com/break/junhong_cmp_fiber/internal/model"
"github.com/break/junhong_cmp_fiber/internal/store/postgres"
"github.com/break/junhong_cmp_fiber/pkg/constants"
)
// TierFacts 是分层轮询匹配所需、卡表之外的业务事实。
type TierFacts struct {
PackageState string // 套餐状态,见 constants.PollingPackageState*
Known bool // 是否成功解析;解析失败时要求套餐状态的配置不匹配
}
type tierCacheEntry struct {
facts TierFacts
loadedAt time.Time
}
// TierResolver 计算卡的套餐状态(绑定设备的卡按设备、独立卡按卡,与停复机"是否有有效套餐"口径一致),
// 并读取运营商账户的流量重置日。结果在进程内短时缓存,层级最迟在缓存过期后的下一次执行时更新。
type TierResolver struct {
usage *postgres.PackageUsageStore
bindings *postgres.DeviceSimBindingStore
carriers *postgres.CarrierStore
logger *zap.Logger
mu sync.Mutex
cache map[uint]tierCacheEntry
resetDays map[uint]int
resetsAt time.Time
}
// NewTierResolver 创建分层事实解析器。
func NewTierResolver(usage *postgres.PackageUsageStore, bindings *postgres.DeviceSimBindingStore, carriers *postgres.CarrierStore, logger *zap.Logger) *TierResolver {
if logger == nil {
logger = zap.NewNop()
}
return &TierResolver{usage: usage, bindings: bindings, carriers: carriers, logger: logger, cache: map[uint]tierCacheEntry{}, resetDays: map[uint]int{}}
}
// Resolve 返回卡的分层事实;缓存未过期时直接返回。
func (r *TierResolver) Resolve(ctx context.Context, card *model.IotCard) TierFacts {
if r == nil || card == nil {
return TierFacts{}
}
if facts, ok := r.cached(card.ID); ok {
return facts
}
r.Prefill(ctx, []*model.IotCard{card})
facts, _ := r.cached(card.ID)
return facts
}
// Prefill 批量解析一组卡的分层事实并写入缓存,供全量初始化等批处理避免逐卡查库。
func (r *TierResolver) Prefill(ctx context.Context, cards []*model.IotCard) {
if r == nil || r.usage == nil || len(cards) == 0 {
return
}
var standaloneIDs, boundCardIDs []uint
for _, card := range cards {
if card == nil {
continue
}
if card.IsStandalone {
standaloneIDs = append(standaloneIDs, card.ID)
} else {
boundCardIDs = append(boundCardIDs, card.ID)
}
}
results := make(map[uint]TierFacts, len(cards))
if len(standaloneIDs) > 0 {
states, err := r.usage.PackageStatesByCarriers(ctx, constants.AssetTypeIotCard, standaloneIDs)
if err != nil {
r.logger.Warn("批量查询卡套餐状态失败,相关卡本次不按套餐状态匹配", zap.Error(err))
}
for _, id := range standaloneIDs {
if err == nil {
results[id] = TierFacts{PackageState: states[id], Known: true}
}
}
}
if len(boundCardIDs) > 0 && r.bindings != nil {
bindings, err := r.bindings.GetActiveBindingsByCardIDs(ctx, boundCardIDs)
if err != nil {
r.logger.Warn("批量查询卡绑定设备失败,相关卡本次不按套餐状态匹配", zap.Error(err))
} else {
deviceOf := make(map[uint]uint, len(bindings))
deviceIDs := make([]uint, 0, len(bindings))
for _, binding := range bindings {
deviceOf[binding.IotCardID] = binding.DeviceID
deviceIDs = append(deviceIDs, binding.DeviceID)
}
deviceStates, devErr := r.usage.PackageStatesByCarriers(ctx, constants.AssetTypeDevice, deviceIDs)
cardStates, cardErr := r.usage.PackageStatesByCarriers(ctx, constants.AssetTypeIotCard, boundCardIDs)
for _, id := range boundCardIDs {
// 有有效设备绑定按设备计算;绑定已失效的卡按卡自身计算(与 resolvePackageCarrier 口径一致)。
if deviceID, bound := deviceOf[id]; bound {
if devErr == nil {
results[id] = TierFacts{PackageState: deviceStates[deviceID], Known: true}
}
} else if cardErr == nil {
results[id] = TierFacts{PackageState: cardStates[id], Known: true}
}
}
}
}
now := time.Now()
r.mu.Lock()
for _, card := range cards {
if card == nil {
continue
}
r.cache[card.ID] = tierCacheEntry{facts: results[card.ID], loadedAt: now}
}
r.mu.Unlock()
}
// Invalidate 清除某卡的分层事实缓存(套餐变化后可调用,未调用时最迟缓存过期后更新)。
func (r *TierResolver) Invalidate(cardID uint) {
if r == nil {
return
}
r.mu.Lock()
delete(r.cache, cardID)
r.mu.Unlock()
}
func (r *TierResolver) cached(cardID uint) (TierFacts, bool) {
r.mu.Lock()
defer r.mu.Unlock()
entry, ok := r.cache[cardID]
if !ok || time.Since(entry.loadedAt) > constants.PollingTierFactsCacheSeconds*time.Second {
return TierFacts{}, false
}
return entry.facts, true
}
// ResetDay 返回运营商账户的流量重置日(1~28),缓存 10 分钟;读取失败按 1 号。
func (r *TierResolver) ResetDay(ctx context.Context, carrierID uint) int {
if r == nil || r.carriers == nil {
return 1
}
r.mu.Lock()
if time.Since(r.resetsAt) > 10*time.Minute {
r.resetDays = map[uint]int{}
r.resetsAt = time.Now()
}
day, ok := r.resetDays[carrierID]
r.mu.Unlock()
if ok {
return day
}
day = r.carriers.GetDataResetDay(ctx, carrierID)
if day < 1 || day > 28 {
day = 1
}
r.mu.Lock()
r.resetDays[carrierID] = day
r.mu.Unlock()
return day
}

View File

@@ -0,0 +1,181 @@
package stopresumerule
import (
"context"
"sort"
"strconv"
"strings"
"github.com/break/junhong_cmp_fiber/internal/domain/stopresume"
"github.com/break/junhong_cmp_fiber/internal/model/dto"
"github.com/break/junhong_cmp_fiber/pkg/constants"
"github.com/break/junhong_cmp_fiber/pkg/errors"
"github.com/break/junhong_cmp_fiber/pkg/middleware"
)
// 明细分页默认值与上限。
const (
defaultPageSize = 20
maxPageSize = 500
)
// EnsureAllowed 校验演练与导出权限:仅超级管理员与平台账号。
func EnsureAllowed(userType int) error {
if userType == constants.UserTypeSuperAdmin || userType == constants.UserTypePlatform {
return nil
}
return errors.New(errors.CodeForbidden, constants.PlatformManagementForbiddenMessage)
}
// BuildFilter 把请求参数转换为演练筛选;假设开启与关闭的账户不得重叠。
func BuildFilter(carrierID *uint, stopReason, assumeOn, assumeOff string) (Filter, error) {
on, err := parseIDs(assumeOn, "assume_control_on")
if err != nil {
return Filter{}, err
}
off, err := parseIDs(assumeOff, "assume_control_off")
if err != nil {
return Filter{}, err
}
control := make(map[uint]bool, len(on)+len(off))
for _, id := range on {
control[id] = true
}
for _, id := range off {
if control[id] {
return Filter{}, errors.New(errors.CodeInvalidParam, "同一运营商账户不能同时假设开启和关闭实名管控")
}
control[id] = false
}
return Filter{CarrierID: carrierID, StopReason: strings.TrimSpace(stopReason), RealnameControl: control}, nil
}
func parseIDs(raw, field string) ([]uint, error) {
raw = strings.TrimSpace(raw)
if raw == "" {
return nil, nil
}
parts := strings.Split(raw, ",")
ids := make([]uint, 0, len(parts))
for _, part := range parts {
part = strings.TrimSpace(part)
if part == "" {
continue
}
id, err := strconv.ParseUint(part, 10, 64)
if err != nil || id == 0 {
return nil, errors.New(errors.CodeInvalidParam, field+" 只能是逗号分隔的运营商账户ID")
}
ids = append(ids, uint(id))
}
return ids, nil
}
type summaryKey struct {
carrierID uint
stopReason string
result string
reason string
}
// DryRun 执行演练:全量分批判断,返回汇总与按 result 筛选后的分页明细。只读。
func (q *Query) DryRun(ctx context.Context, req *dto.StopResumeRuleDryRunRequest) (*dto.StopResumeRuleDryRunResponse, error) {
if err := EnsureAllowed(middleware.GetUserTypeFromContext(ctx)); err != nil {
return nil, err
}
filter, err := BuildFilter(req.CarrierID, req.StopReason, req.AssumeControlOn, req.AssumeControlOff)
if err != nil {
return nil, err
}
page, size := req.Page, req.PageSize
if page < 1 {
page = 1
}
if size < 1 {
size = defaultPageSize
}
if size > maxPageSize {
size = maxPageSize
}
offset := (page - 1) * size
resp := &dto.StopResumeRuleDryRunResponse{Page: page, PageSize: size, Items: []dto.StopResumeRuleDryRunItem{}}
counts := map[summaryKey]int64{}
names := map[uint]string{}
var matched int64
err = q.Scan(ctx, filter, func(rows []Row) error {
for index := range rows {
row := &rows[index]
resp.Totals.Total++
switch row.Result {
case stopresume.ResultWillResume:
resp.Totals.WillResume++
case stopresume.ResultWillStop:
resp.Totals.WillStop++
default:
resp.Totals.NoAction++
}
counts[summaryKey{row.CarrierID, row.StopReason, row.Result, row.Reason}]++
names[row.CarrierID] = row.CarrierName
if req.Result != "" && row.Result != req.Result {
continue
}
if matched >= int64(offset) && len(resp.Items) < size {
resp.Items = append(resp.Items, ToItem(row))
}
matched++
}
return nil
})
if err != nil {
return nil, err
}
resp.Total = matched
resp.Summary = buildSummary(counts, names)
return resp, nil
}
func buildSummary(counts map[summaryKey]int64, names map[uint]string) []dto.StopResumeRuleDryRunSummaryItem {
items := make([]dto.StopResumeRuleDryRunSummaryItem, 0, len(counts))
for key, count := range counts {
items = append(items, dto.StopResumeRuleDryRunSummaryItem{
CarrierID: key.carrierID, CarrierName: names[key.carrierID],
StopReason: key.stopReason, StopReasonName: stopReasonName(key.stopReason),
Result: key.result, ResultName: stopresume.ResultName(key.result),
Reason: key.reason, ReasonName: stopresume.ReasonName(key.reason), Count: count,
})
}
sort.Slice(items, func(i, j int) bool {
a, b := items[i], items[j]
if a.CarrierID != b.CarrierID {
return a.CarrierID < b.CarrierID
}
if a.StopReason != b.StopReason {
return a.StopReason < b.StopReason
}
if a.Result != b.Result {
return a.Result < b.Result
}
return a.Reason < b.Reason
})
return items
}
// ToItem 把演练行转换为接口明细。
func ToItem(row *Row) dto.StopResumeRuleDryRunItem {
return dto.StopResumeRuleDryRunItem{
CardID: row.CardID, ICCID: row.ICCID, CarrierID: row.CarrierID, CarrierName: row.CarrierName,
DeviceID: row.DeviceID, NetworkStatus: row.NetworkStatus,
StopReason: row.StopReason, StopReasonName: stopReasonName(row.StopReason),
CardCategory: row.CardCategory, RealnameOK: row.RealnameOK,
Result: row.Result, ResultName: stopresume.ResultName(row.Result),
Reason: row.Reason, ReasonName: stopresume.ReasonName(row.Reason),
}
}
// stopReasonName 返回当前停机原因名称;空停因显示为"无"。
func stopReasonName(reason string) string {
if strings.TrimSpace(reason) == "" {
return "无"
}
return constants.AgentOpenAPIStopReasonName(reason)
}

View File

@@ -0,0 +1,329 @@
// Package stopresumerule 提供停复机规则演练的只读查询:按批读取卡与资产当前事实,按新规则判断目标动作。
//
// 演练不调用网关、不写任何卡、指令或调度状态;始终按新规则判断,不受停复机总开关影响;
// 可传入假设的通道实名管控配置预演,实际配置不变。
package stopresumerule
import (
"context"
"time"
"gorm.io/gorm"
carrierthresholddomain "github.com/break/junhong_cmp_fiber/internal/domain/carrierthreshold"
"github.com/break/junhong_cmp_fiber/internal/domain/stopresume"
"github.com/break/junhong_cmp_fiber/internal/model"
"github.com/break/junhong_cmp_fiber/pkg/constants"
"github.com/break/junhong_cmp_fiber/pkg/errors"
)
// batchSize 是演练每批读取的卡数。
const batchSize = 1000
// StopReasonEmpty 是筛选"停机原因为空"时使用的取值。
const StopReasonEmpty = "none"
// Filter 是演练筛选条件。
type Filter struct {
CarrierID *uint
StopReason string // 为空不限;StopReasonEmpty 表示停机原因为空
// RealnameControl 假设的通道实名管控:键为运营商账户 ID,值为是否开启;未列出的账户按实际配置。
RealnameControl map[uint]bool
}
// Row 是一张卡的演练结果。
type Row struct {
CardID uint
ICCID string
CarrierID uint
CarrierName string
DeviceID uint // 无有效绑定为 0
NetworkStatus int
StopReason string
CardCategory string
RealnameOK bool
stopresume.Decision
}
// Query 停复机规则演练查询。
type Query struct {
db *gorm.DB
now func() time.Time
}
// NewQuery 创建演练查询。
func NewQuery(db *gorm.DB) *Query {
return &Query{db: db, now: time.Now}
}
// Scan 按卡 ID 升序分批演练全部符合筛选的开机与停机卡,每批回调一次。
func (q *Query) Scan(ctx context.Context, filter Filter, fn func([]Row) error) error {
carriers, err := q.loadCarriers(ctx)
if err != nil {
return err
}
var lastID uint
for {
if err := ctx.Err(); err != nil {
return err
}
var cards []model.IotCard
if err := q.cardQuery(ctx, filter).Where("id > ?", lastID).Order("id ASC").Limit(batchSize).Find(&cards).Error; err != nil {
return errors.Wrap(errors.CodeDatabaseError, err, "查询演练卡失败")
}
if len(cards) == 0 {
return nil
}
lastID = cards[len(cards)-1].ID
rows, err := q.evaluate(ctx, cards, carriers, filter)
if err != nil {
return err
}
if err := fn(rows); err != nil {
return err
}
}
}
// Count 统计符合筛选的卡数(导出行数)。
func (q *Query) Count(ctx context.Context, filter Filter) (int64, error) {
var total int64
if err := q.cardQuery(ctx, filter).Count(&total).Error; err != nil {
return 0, errors.Wrap(errors.CodeDatabaseError, err, "统计演练卡数失败")
}
return total, nil
}
// Page 按卡 ID 升序取 offset/limit 范围内的卡并演练,供导出分片使用。
func (q *Query) Page(ctx context.Context, filter Filter, offset, limit int) ([]Row, error) {
carriers, err := q.loadCarriers(ctx)
if err != nil {
return nil, err
}
var cards []model.IotCard
if err := q.cardQuery(ctx, filter).Order("id ASC").Offset(offset).Limit(limit).Find(&cards).Error; err != nil {
return nil, errors.Wrap(errors.CodeDatabaseError, err, "查询演练卡失败")
}
return q.evaluate(ctx, cards, carriers, filter)
}
func (q *Query) cardQuery(ctx context.Context, filter Filter) *gorm.DB {
query := q.db.WithContext(ctx).Model(&model.IotCard{}).
Select("id", "iccid", "carrier_id", "network_status", "stop_reason", "enable_polling", "real_name_status",
"card_category", "is_standalone").
Where("network_status IN ?", []int{constants.NetworkStatusOffline, constants.NetworkStatusOnline})
if filter.CarrierID != nil {
query = query.Where("carrier_id = ?", *filter.CarrierID)
}
switch filter.StopReason {
case "":
case StopReasonEmpty:
query = query.Where("COALESCE(stop_reason, '') = ''")
default:
query = query.Where("stop_reason = ?", filter.StopReason)
}
return query
}
type carrierFacts struct {
name string
realnameControl bool
resetDay int
}
func (q *Query) loadCarriers(ctx context.Context) (map[uint]carrierFacts, error) {
var carriers []model.Carrier
if err := q.db.WithContext(ctx).Select("id", "carrier_name", "realname_control_enabled", "data_reset_day").
Find(&carriers).Error; err != nil {
return nil, errors.Wrap(errors.CodeDatabaseError, err, "查询运营商账户失败")
}
result := make(map[uint]carrierFacts, len(carriers))
for _, carrier := range carriers {
result[carrier.ID] = carrierFacts{
name: carrier.CarrierName, realnameControl: carrier.RealnameControlEnabled == 1, resetDay: carrier.DataResetDay,
}
}
return result, nil
}
// evaluate 批量加载一批卡的绑定、设备轮询开关、套餐与通道阈值锁,再逐卡判断。
func (q *Query) evaluate(ctx context.Context, cards []model.IotCard, carriers map[uint]carrierFacts, filter Filter) ([]Row, error) {
if len(cards) == 0 {
return nil, nil
}
cardIDs := make([]uint, 0, len(cards))
for _, card := range cards {
cardIDs = append(cardIDs, card.ID)
}
deviceOf, err := q.activeBindings(ctx, cards)
if err != nil {
return nil, err
}
deviceIDs := uniqueValues(deviceOf)
devicePolling, err := q.devicePolling(ctx, deviceIDs)
if err != nil {
return nil, err
}
cardPackages, err := q.packages(ctx, "iot_card_id", cardIDs)
if err != nil {
return nil, err
}
devicePackages, err := q.packages(ctx, "device_id", deviceIDs)
if err != nil {
return nil, err
}
locked, err := q.lockedCards(ctx, cardIDs, carriers)
if err != nil {
return nil, err
}
rows := make([]Row, 0, len(cards))
for _, card := range cards {
deviceID := deviceOf[card.ID]
facts := packageFactsOf(cardPackages[card.ID])
suspended := !card.EnablePolling
if deviceID != 0 {
facts = packageFactsOf(devicePackages[deviceID])
if enabled, ok := devicePolling[deviceID]; ok && !enabled {
suspended = true
}
}
carrier, known := carriers[card.CarrierID]
control := carrier.realnameControl
if assumed, ok := filter.RealnameControl[card.CarrierID]; ok {
control = assumed
}
verified := card.RealNameStatus == constants.RealNameStatusVerified
decision := stopresume.Decide(stopresume.Facts{
NetworkStatus: card.NetworkStatus, PollingSuspended: suspended,
HasValidPackage: facts.hasValid, TrafficExhausted: facts.exhausted,
RealnameVerified: verified, CarrierKnown: known, RealnameControl: control,
ThresholdLocked: locked[card.ID], CurrentStopReason: card.StopReason,
})
rows = append(rows, Row{
CardID: card.ID, ICCID: card.ICCID, CarrierID: card.CarrierID, CarrierName: carrier.name, DeviceID: deviceID,
NetworkStatus: card.NetworkStatus, StopReason: card.StopReason, CardCategory: card.CardCategory,
RealnameOK: !control || verified, Decision: decision,
})
}
return rows, nil
}
// activeBindings 返回非独立卡的有效绑定设备;与停复机评估一致,独立卡不查绑定。
func (q *Query) activeBindings(ctx context.Context, cards []model.IotCard) (map[uint]uint, error) {
bound := make([]uint, 0, len(cards))
for _, card := range cards {
if !card.IsStandalone {
bound = append(bound, card.ID)
}
}
result := make(map[uint]uint, len(bound))
if len(bound) == 0 {
return result, nil
}
var bindings []model.DeviceSimBinding
if err := q.db.WithContext(ctx).Select("iot_card_id", "device_id").
Where("iot_card_id IN ? AND bind_status = ?", bound, constants.BindStatusBound).
Find(&bindings).Error; err != nil {
return nil, errors.Wrap(errors.CodeDatabaseError, err, "查询卡绑定设备失败")
}
for _, binding := range bindings {
result[binding.IotCardID] = binding.DeviceID
}
return result, nil
}
func (q *Query) devicePolling(ctx context.Context, deviceIDs []uint) (map[uint]bool, error) {
result := make(map[uint]bool, len(deviceIDs))
if len(deviceIDs) == 0 {
return result, nil
}
var devices []model.Device
if err := q.db.WithContext(ctx).Select("id", "enable_polling").Where("id IN ?", deviceIDs).Find(&devices).Error; err != nil {
return nil, errors.Wrap(errors.CodeDatabaseError, err, "查询设备轮询状态失败")
}
for _, device := range devices {
result[device.ID] = device.EnablePolling
}
return result, nil
}
// packages 读取载体生效中与已用完的套餐,按载体分组。
func (q *Query) packages(ctx context.Context, column string, ids []uint) (map[uint][]model.PackageUsage, error) {
result := make(map[uint][]model.PackageUsage, len(ids))
if len(ids) == 0 {
return result, nil
}
var usages []model.PackageUsage
if err := q.db.WithContext(ctx).Where(column+" IN ?", ids).
Where("status IN ?", []int{constants.PackageUsageStatusActive, constants.PackageUsageStatusDepleted}).
Find(&usages).Error; err != nil {
return nil, errors.Wrap(errors.CodeDatabaseError, err, "查询套餐使用记录失败")
}
for _, usage := range usages {
key := usage.IotCardID
if column == "device_id" {
key = usage.DeviceID
}
result[key] = append(result[key], usage)
}
return result, nil
}
type packageFacts struct {
hasValid bool
exhausted bool
}
// packageFactsOf 与评估主流程同口径:有生效中套餐即有套餐;生效或已用完的套餐全部达到上限才算流量用完。
func packageFactsOf(usages []model.PackageUsage) packageFacts {
facts := packageFacts{}
if len(usages) == 0 {
return facts
}
facts.exhausted = true
for index := range usages {
if usages[index].Status == constants.PackageUsageStatusActive {
facts.hasValid = true
}
if !usages[index].IsTrafficDepleted() {
facts.exhausted = false
}
}
return facts
}
// lockedCards 返回持本周期通道阈值锁的卡;与停复机评估同口径,账户不存在或重置日非法时按仍生效处理。
func (q *Query) lockedCards(ctx context.Context, cardIDs []uint, carriers map[uint]carrierFacts) (map[uint]bool, error) {
result := make(map[uint]bool)
var locks []model.CarrierTrafficThresholdLock
if err := q.db.WithContext(ctx).Select("card_id", "carrier_id", "period_start").
Where("card_id IN ? AND status = ?", cardIDs, carrierthresholddomain.LockStatusLocked).
Find(&locks).Error; err != nil {
return nil, errors.Wrap(errors.CodeDatabaseError, err, "查询通道阈值锁失败")
}
now := q.now()
for _, lock := range locks {
carrier, ok := carriers[lock.CarrierID]
if !ok {
result[lock.CardID] = true
continue
}
periodStart, err := carrierthresholddomain.PeriodStart(now, carrier.resetDay)
if err != nil || lock.PeriodStart.Equal(periodStart) {
result[lock.CardID] = true
}
}
return result, nil
}
func uniqueValues(m map[uint]uint) []uint {
seen := make(map[uint]bool, len(m))
result := make([]uint, 0, len(m))
for _, value := range m {
if value != 0 && !seen[value] {
seen[value] = true
result = append(result, value)
}
}
return result
}

View File

@@ -112,6 +112,9 @@ func RegisterAdminRoutes(router fiber.Router, handlers *bootstrap.Handlers, midd
if handlers.AssetAutoRenewal != nil {
registerAssetAutoRenewalRoutes(authGroup, handlers.AssetAutoRenewal, doc, basePath)
}
if handlers.StopResumeRule != nil {
registerStopResumeRuleRoutes(authGroup, handlers.StopResumeRule, doc, basePath)
}
if handlers.OperationsReport != nil {
registerOperationsReportRoutes(authGroup, handlers.OperationsReport, doc, basePath)
}
@@ -133,6 +136,12 @@ func RegisterAdminRoutes(router fiber.Router, handlers *bootstrap.Handlers, midd
if handlers.PollingConfig != nil {
registerPollingConfigRoutes(authGroup, handlers.PollingConfig, doc, basePath)
}
if handlers.CarrierScheduler != nil {
registerCarrierSchedulerRoutes(authGroup, handlers.CarrierScheduler, doc, basePath)
}
if handlers.GatewayRateLimit != nil {
registerGatewayRateLimitRoutes(authGroup, handlers.GatewayRateLimit, doc, basePath)
}
if handlers.PollingConcurrency != nil {
registerPollingConcurrencyRoutes(authGroup, handlers.PollingConcurrency, doc, basePath)
}

View File

@@ -70,10 +70,13 @@ func registerCarrierRoutes(router fiber.Router, handler *admin.CarrierHandler, d
Register(carriers, doc, groupPath, "PUT", "/:id", handler.Update, RouteSpec{
Summary: "更新运营商",
Tags: []string{"运营商管理"},
Input: new(dto.UpdateCarrierParams),
Output: new(dto.CarrierResponse),
Auth: true,
Description: "通道流量阈值与通道实名管控(realname_control_enabled)仅超级管理员与平台账号可修改,非平台账号携带即整体拒绝;" +
"实名管控从 0 改为 1 时必须携带 realname_control_confirm=true,未确认时返回该通道未实名开机卡数且不保存;" +
"修改并入运营商更新审计",
Tags: []string{"运营商管理"},
Input: new(dto.UpdateCarrierParams),
Output: new(dto.CarrierResponse),
Auth: true,
})
Register(carriers, doc, groupPath, "DELETE", "/:id", handler.Delete, RouteSpec{

View File

@@ -0,0 +1,68 @@
package routes
import (
"github.com/gofiber/fiber/v2"
"github.com/break/junhong_cmp_fiber/internal/handler/admin"
"github.com/break/junhong_cmp_fiber/internal/model/dto"
"github.com/break/junhong_cmp_fiber/pkg/openapi"
)
// registerCarrierSchedulerRoutes 注册按运营商账户调度的后台接口。读取仅超级管理员与平台账号,修改仅超级管理员(服务层校验)。
func registerCarrierSchedulerRoutes(router fiber.Router, handler *admin.CarrierSchedulerHandler, doc *openapi.Generator, basePath string) {
tags := []string{"轮询调度"}
params := router.Group("/polling-carrier-params")
paramsPath := basePath + "/polling-carrier-params"
Register(params, doc, paramsPath, "GET", "", handler.ListParams, RouteSpec{
Summary: "查询运营商账户调度参数列表", Description: "未单独配置的账户返回全局默认并标记 is_default。", Tags: tags,
Input: nil, Output: new(dto.CarrierPollingParamListResponse), Auth: true,
})
Register(params, doc, paramsPath, "GET", "/:carrier_id", handler.GetParam, RouteSpec{
Summary: "查询运营商账户调度参数", Tags: tags, Input: new(dto.CarrierPollingParamPath), Output: new(dto.CarrierPollingParamItem), Auth: true,
})
Register(params, doc, paramsPath, "PUT", "/:carrier_id", handler.SaveParam, RouteSpec{
Summary: "保存运营商账户调度参数(仅超级管理员)",
Description: "字段为空表示该项使用全局默认;执行上限不得大于当前订阅进程 max_in_flight 之和;下一轮调度生效并写统一审计。",
Tags: tags, Input: new(dto.SaveCarrierPollingParamRequest), Output: new(dto.CarrierPollingParamItem), Auth: true,
})
Register(params, doc, paramsPath, "DELETE", "/:carrier_id", handler.DeleteParam, RouteSpec{
Summary: "删除运营商账户调度参数(恢复全局默认,仅超级管理员)", Tags: tags, Input: new(dto.CarrierPollingParamPath), Output: nil, Auth: true,
})
scenes := router.Group("/polling-check-scenes")
scenesPath := basePath + "/polling-check-scenes"
Register(scenes, doc, scenesPath, "GET", "", handler.ListScenes, RouteSpec{
Summary: "查询请求检查场景", Description: "业务事件与读入口在按运营商账户调度时按场景请求检查;场景由代码预置。", Tags: tags,
Input: nil, Output: new([]dto.PollingCheckSceneItem), Auth: true,
})
Register(scenes, doc, scenesPath, "PUT", "/:scene", handler.UpdateScene, RouteSpec{
Summary: "修改请求检查场景(仅超级管理员)", Description: "只可修改检查类型、延迟确认序列(递增)与冷却时长。", Tags: tags,
Input: new(dto.UpdatePollingCheckSceneRequest), Output: new(dto.PollingCheckSceneItem), Auth: true,
})
alerts := router.Group("/polling-failure-alerts")
alertsPath := basePath + "/polling-failure-alerts"
Register(alerts, doc, alertsPath, "GET", "", handler.ListAlerts, RouteSpec{
Summary: "查询卡轮询失败告警列表", Tags: tags, Input: new(dto.PollingFailureAlertListRequest), Output: new(dto.PollingFailureAlertPageResult), Auth: true,
})
Register(alerts, doc, alertsPath, "GET", "/summary", handler.AlertSummary, RouteSpec{
Summary: "未处理卡轮询失败告警按运营商账户与错误类别汇总", Tags: tags, Input: nil, Output: new([]dto.PollingFailureAlertSummaryItem), Auth: true,
})
Register(alerts, doc, alertsPath, "GET", "/:id", handler.GetAlert, RouteSpec{
Summary: "查询卡轮询失败告警详情", Tags: tags, Input: new(dto.PollingFailureAlertIDPath), Output: new(dto.PollingFailureAlertItem), Auth: true,
})
Register(alerts, doc, alertsPath, "PUT", "/:id/status", handler.HandleAlert, RouteSpec{
Summary: "标记卡轮询失败告警已处理", Description: "只有未处理告警可标记;关闭该卡轮询请使用资产轮询开关接口。", Tags: tags,
Input: new(dto.HandlePollingFailureAlertRequest), Output: nil, Auth: true,
})
Register(router, doc, basePath, "GET", "/iot-cards/:id/polling-state", handler.CardState, RouteSpec{
Summary: "查询单卡轮询状态", Description: "各检查类型下次到期、最近成功、连续失败与最近错误、是否执行中;只读。", Tags: tags,
Input: new(dto.CardPollingStatePath), Output: new(dto.CardPollingStateResponse), Auth: true,
})
Register(router, doc, basePath, "GET", "/polling-stats/carrier-queues", handler.CarrierStats, RouteSpec{
Summary: "按运营商账户的轮询队列与执行监控", Description: "按运营商账户调度的各账户队列长度、已到期数、执行中、上一分钟 QPS、限流等待与拒绝、处理中标记到期、未处理告警。", Tags: tags,
Input: nil, Output: new(dto.CarrierPollingStatsResponse), Auth: true,
})
}

View File

@@ -6,24 +6,15 @@ import (
"github.com/break/junhong_cmp_fiber/internal/gateway"
"github.com/break/junhong_cmp_fiber/internal/handler/admin"
"github.com/break/junhong_cmp_fiber/internal/model/dto"
"github.com/break/junhong_cmp_fiber/pkg/constants"
"github.com/break/junhong_cmp_fiber/pkg/errors"
"github.com/break/junhong_cmp_fiber/pkg/middleware"
"github.com/break/junhong_cmp_fiber/pkg/openapi"
)
func registerDeviceRoutes(router fiber.Router, handler *admin.DeviceHandler, importHandler *admin.DeviceImportHandler, doc *openapi.Generator, basePath string) {
devices := router.Group("/devices")
groupPath := basePath + "/devices"
platformDevices := router.Group("/devices", func(c *fiber.Ctx) error {
userType := middleware.GetUserTypeFromContext(c.UserContext())
if userType != constants.UserTypeSuperAdmin && userType != constants.UserTypePlatform {
return errors.New(errors.CodeForbidden, constants.PlatformManagementForbiddenMessage)
}
return c.Next()
})
Register(platformDevices, doc, groupPath, "GET", "/types", handler.ListDeviceTypes, RouteSpec{
// 仅平台可用的判断放在 ListDeviceTypes 内部:Fiber 的 Group 带处理函数会注册为整个 /devices 前缀的中间件,会拦住代理与企业账号的所有设备接口。
Register(devices, doc, groupPath, "GET", "/types", handler.ListDeviceTypes, RouteSpec{
Summary: "查询设备类型选项",
Description: "仅超级管理员与平台账号可访问。返回未删除设备中去重、去除首尾空白且按升序排列的非空设备类型字符串。",
Tags: []string{"设备管理"},

View File

@@ -0,0 +1,72 @@
package routes
import (
"github.com/gofiber/fiber/v2"
"github.com/break/junhong_cmp_fiber/internal/handler/admin"
"github.com/break/junhong_cmp_fiber/internal/model/dto"
"github.com/break/junhong_cmp_fiber/pkg/openapi"
)
// registerGatewayRateLimitRoutes 注册运营商账户网关限流、错误分类、停复机记录与调用统计路由。
func registerGatewayRateLimitRoutes(router fiber.Router, handler *admin.GatewayRateLimitHandler, doc *openapi.Generator, basePath string) {
tags := []string{"网关限流"}
groups := router.Group("/polling-rate-limit-groups")
groupsPath := basePath + "/polling-rate-limit-groups"
Register(groups, doc, groupsPath, "GET", "", handler.ListGroups, RouteSpec{
Summary: "查询网关限流组列表", Tags: tags, Input: new(dto.RateLimitGroupListRequest), Output: new(dto.RateLimitGroupPageResult), Auth: true,
})
Register(groups, doc, groupsPath, "POST", "", handler.CreateGroup, RouteSpec{
Summary: "新建网关限流组(仅超级管理员)", Tags: tags, Input: new(dto.SaveRateLimitGroupRequest), Output: new(dto.RateLimitGroupResponse), Auth: true,
})
Register(groups, doc, groupsPath, "GET", "/:id", handler.GetGroup, RouteSpec{
Summary: "查询网关限流组详情", Tags: tags, Input: new(dto.IDReq), Output: new(dto.RateLimitGroupResponse), Auth: true,
})
Register(groups, doc, groupsPath, "PUT", "/:id", handler.UpdateGroup, RouteSpec{
Summary: "修改网关限流组(仅超级管理员)", Tags: tags, Input: new(dto.UpdateRateLimitGroupParams), Output: new(dto.RateLimitGroupResponse), Auth: true,
})
Register(groups, doc, groupsPath, "PUT", "/:id/status", handler.UpdateGroupStatus, RouteSpec{
Summary: "启停网关限流组(仅超级管理员)", Tags: tags, Input: new(dto.UpdateRateLimitGroupStatusRequest), Output: nil, Auth: true,
})
Register(groups, doc, groupsPath, "DELETE", "/:id", handler.DeleteGroup, RouteSpec{
Summary: "删除网关限流组(仅超级管理员)", Tags: tags, Input: new(dto.IDReq), Output: nil, Auth: true,
})
rules := router.Group("/gateway-error-classifications")
rulesPath := basePath + "/gateway-error-classifications"
Register(rules, doc, rulesPath, "GET", "", handler.ListRules, RouteSpec{
Summary: "查询上游错误分类规则列表", Tags: tags, Input: new(dto.GatewayErrorRuleListRequest), Output: new(dto.GatewayErrorRulePageResult), Auth: true,
})
Register(rules, doc, rulesPath, "POST", "", handler.CreateRule, RouteSpec{
Summary: "新建上游错误分类规则(仅超级管理员)", Tags: tags, Input: new(dto.SaveGatewayErrorRuleRequest), Output: new(dto.GatewayErrorRuleResponse), Auth: true,
})
Register(rules, doc, rulesPath, "POST", "/test", handler.TestRule, RouteSpec{
Summary: "测试上游错误分类(不写数据)", Tags: tags, Input: new(dto.TestGatewayErrorRuleRequest), Output: new(dto.TestGatewayErrorRuleResponse), Auth: true,
})
Register(rules, doc, rulesPath, "PUT", "/:id", handler.UpdateRule, RouteSpec{
Summary: "修改上游错误分类规则(仅超级管理员)", Tags: tags, Input: new(dto.UpdateGatewayErrorRuleParams), Output: new(dto.GatewayErrorRuleResponse), Auth: true,
})
Register(rules, doc, rulesPath, "PUT", "/:id/status", handler.UpdateRuleStatus, RouteSpec{
Summary: "启停上游错误分类规则(仅超级管理员)", Tags: tags, Input: new(dto.UpdateGatewayErrorRuleStatusRequest), Output: nil, Auth: true,
})
Register(rules, doc, rulesPath, "DELETE", "/:id", handler.DeleteRule, RouteSpec{
Summary: "删除上游错误分类规则(仅超级管理员)", Tags: tags, Input: new(dto.IDReq), Output: nil, Auth: true,
})
Register(router, doc, basePath, "GET", "/card-stop-resume-records", handler.ListStopResumeRecords, RouteSpec{
Summary: "查询停复机记录(时间范围必填,跨度不超过 31 天)", Tags: tags, Input: new(dto.StopResumeRecordListRequest), Output: new(dto.StopResumeRecordPageResult), Auth: true,
})
Register(router, doc, basePath, "GET", "/iot-cards/:id/stop-resume-records", handler.ListCardStopResumeRecords, RouteSpec{
Summary: "查询单卡停复机历史", Tags: tags, Input: new(dto.CardStopResumeRecordListRequest), Output: new(dto.StopResumeRecordPageResult), Auth: true,
})
Register(router, doc, basePath, "GET", "/polling-stats/carriers", handler.CarrierCallStats, RouteSpec{
Summary: "按运营商账户汇总网关调用", Tags: tags, Input: new(dto.GatewayCallStatsRequest), Output: new(dto.GatewayCallStatsResponse), Auth: true,
})
Register(router, doc, basePath, "GET", "/polling-stats/carriers/:carrier_id/trend", handler.CarrierCallTrend, RouteSpec{
Summary: "查询单个运营商账户的网关调用趋势", Tags: tags, Input: new(dto.GatewayCallTrendRequest), Output: new(dto.GatewayCallTrendResponse), Auth: true,
})
Register(router, doc, basePath, "GET", "/polling-options", handler.PollingOptions, RouteSpec{
Summary: "查询轮询与网关限流枚举", Tags: tags, Input: nil, Output: new(dto.PollingOptionsResponse), Auth: true,
})
}

View File

@@ -37,6 +37,62 @@ func registerPollingConfigRoutes(router fiber.Router, handler *admin.PollingConf
Auth: true,
})
Register(configs, doc, groupPath, "PUT", "/priorities", handler.UpdatePriorities, RouteSpec{
Summary: "整体调整轮询配置优先级(仅超级管理员)",
Tags: []string{"轮询配置管理"},
Input: new(dto.UpdatePollingConfigPrioritiesRequest),
Output: nil,
Auth: true,
})
Register(configs, doc, groupPath, "GET", "/match-preview", handler.MatchPreview, RouteSpec{
Summary: "预览卡的轮询配置匹配结果(按当前配置)",
Tags: []string{"轮询配置管理"},
Input: new(dto.PollingConfigMatchPreviewQuery),
Output: new(dto.PollingConfigMatchPreviewResponse),
Auth: true,
})
Register(configs, doc, groupPath, "POST", "/match-preview", handler.MatchPreview, RouteSpec{
Summary: "预览卡的轮询配置匹配结果(可带未保存修改,不写数据)",
Tags: []string{"轮询配置管理"},
Input: new(dto.PollingConfigMatchPreviewRequest),
Output: new(dto.PollingConfigMatchPreviewResponse),
Auth: true,
})
Register(configs, doc, groupPath, "POST", "/recalculate-due", handler.CreateRecalculateTask, RouteSpec{
Summary: "立即重算轮询到期时间(异步,只提前不推后,仅超级管理员)",
Tags: []string{"轮询配置管理"},
Input: new(dto.PollingRecalculateDueRequest),
Output: new(dto.PollingRecalculateDueResponse),
Auth: true,
})
Register(configs, doc, groupPath, "GET", "/recalculate-due/:task_id", handler.GetRecalculateTask, RouteSpec{
Summary: "查询重算轮询到期时间任务进度",
Tags: []string{"轮询配置管理"},
Input: new(dto.PollingRecalculateDueTaskIDRequest),
Output: new(dto.PollingRecalculateDueResponse),
Auth: true,
})
Register(router, doc, basePath, "GET", "/polling-stats/pre-reset", handler.PreResetStats, RouteSpec{
Summary: "重置日前加密查流量效果统计",
Tags: []string{"轮询配置管理"},
Input: nil,
Output: new(dto.PollingPreResetStatsResponse),
Auth: true,
})
Register(router, doc, basePath, "GET", "/polling-stats/tiers", handler.TierStats, RouteSpec{
Summary: "分层轮询统计(结果缓存 5 分钟)",
Tags: []string{"轮询配置管理"},
Input: nil,
Output: new(dto.PollingTierStatsResponse),
Auth: true,
})
Register(configs, doc, groupPath, "GET", "/:id", handler.Get, RouteSpec{
Summary: "获取轮询配置详情",
Tags: []string{"轮询配置管理"},

Some files were not shown because too many files have changed in this diff Show More