This commit is contained in:
98
docs/add-asset-operation-audit-log/功能总结.md
Normal file
98
docs/add-asset-operation-audit-log/功能总结.md
Normal file
@@ -0,0 +1,98 @@
|
||||
# 资产操作审计日志功能总结
|
||||
|
||||
## 1. 功能目标
|
||||
|
||||
为资产域(IoT 卡/设备)补齐统一审计落库能力,解决“谁在什么时间对哪个资产做了什么变更、结果如何”的追溯问题。
|
||||
|
||||
## 2. 数据模型与落库
|
||||
|
||||
- 新增表:`tb_asset_operation_log`
|
||||
- 关键字段:
|
||||
- 操作人:`operator_id/operator_type/operator_name`
|
||||
- 资产:`asset_type/asset_id/asset_identifier`
|
||||
- 操作:`operation_type/operation_desc`
|
||||
- 镜像:`before_data/after_data`(JSONB)
|
||||
- 请求上下文:`request_id/ip_address/user_agent/request_path/request_method`
|
||||
- 结果:`result_status/error_code/error_msg`
|
||||
- 批量统计:`batch_total/success_count/fail_count`
|
||||
- 索引:
|
||||
- `idx_asset_log_asset_created`
|
||||
- `idx_asset_log_identifier_created`
|
||||
- `idx_asset_log_operator_created`
|
||||
- `idx_asset_log_operation_created`
|
||||
- `idx_asset_log_result_created`
|
||||
|
||||
## 3. 统一封装设计
|
||||
|
||||
### 3.1 基础设施统一入口
|
||||
|
||||
- `internal/service/asset_audit/service.go`
|
||||
- 统一 `LogOperation(ctx, log)` 异步写入
|
||||
- 写入失败只打 Error 日志,不阻断业务
|
||||
- `internal/service/asset_audit/builder.go`
|
||||
- 统一组装 `before/after`、请求上下文、错误码摘要
|
||||
- 统一脱敏与裁剪
|
||||
|
||||
### 3.2 业务侧统一 helper
|
||||
|
||||
- IoT 卡:`internal/service/iot_card/audit.go`
|
||||
- 设备:`internal/service/device/audit.go`
|
||||
- 统一资产入口轮询:`internal/service/polling/asset_polling_service.go` 内部 helper
|
||||
- 资产停用:`internal/service/asset/lifecycle_service.go` 内部 helper
|
||||
- 导入任务:
|
||||
- `internal/service/device_import/audit.go`
|
||||
- `internal/service/iot_card_import/audit.go`
|
||||
|
||||
业务函数只传结构化参数,避免日志拼装散落在各分支。
|
||||
|
||||
## 4. 覆盖范围
|
||||
|
||||
### 4.1 IoT 卡
|
||||
|
||||
- 分配/回收、系列绑定、轮询开关、实名策略、手动实名状态、删除/批量删除
|
||||
- 自动停复机 + 手动停复机(含 denied/failed)
|
||||
|
||||
### 4.2 设备
|
||||
|
||||
- 删除、分配/回收、系列绑定、绑卡/解绑
|
||||
- 设备停机/复机(含成功数、失败数、失败摘要)
|
||||
- 远程控制:限速、WiFi、切卡、切卡模式、重启、恢复出厂
|
||||
|
||||
### 4.3 统一资产入口与导入
|
||||
|
||||
- 统一入口:停用、轮询状态、实名策略
|
||||
- 导入任务创建:设备导入、卡导入
|
||||
|
||||
## 5. 脱敏与体积控制
|
||||
|
||||
- 敏感键脱敏:`password/passwd/pwd/secret/token/wifi_password/wifipwd`
|
||||
- WiFi 场景使用 `password` 键传参,落库前统一掩码处理
|
||||
- `before_data/after_data` 超过阈值时裁剪并标记 `_truncated=true`
|
||||
|
||||
## 6. 查询与排障示例
|
||||
|
||||
### 6.1 审计日志查询 API(前端)
|
||||
|
||||
- 路径:`GET /api/admin/assets/:identifier/operation-logs`
|
||||
- 操作人相关字段:
|
||||
- `operator_type` 返回中文标准文案(如平台用户、代理账号、企业账号、个人客户、系统)
|
||||
- `operator_type_code` 保留底层枚举编码(如 `admin_user`、`agent_user`)
|
||||
- `operator_name` 优先返回可读名称,历史 `类型#ID` 占位值会在查询时尽量补全
|
||||
- 建议前端优先使用:
|
||||
- `operation_content_before`(操作内容变更前)
|
||||
- `operation_content_after`(操作内容变更后)
|
||||
- `operation_fields_desc`(字段中文说明)
|
||||
- 原始字段 `before_data/after_data` 建议仅用于调试回放。
|
||||
- 字段明细文档:`docs/add-asset-operation-audit-log/操作内容字段说明.md`
|
||||
|
||||
### 6.2 SQL 排障示例
|
||||
|
||||
完整 SQL 见:
|
||||
|
||||
- `docs/add-asset-operation-audit-log/手工验收脚本.sql`
|
||||
|
||||
常用排障问题:
|
||||
|
||||
1. 某资产最近操作:按 `asset_type + asset_id` 查询
|
||||
2. 某操作人近期高风险动作:按 `operator_id` + `result_status in ('failed','denied')`
|
||||
3. WiFi 明文泄露检查:检索 `after_data::text ILIKE '%\"password\":\"%'` 并人工确认是否掩码
|
||||
89
docs/add-asset-operation-audit-log/手工验收脚本.sql
Normal file
89
docs/add-asset-operation-audit-log/手工验收脚本.sql
Normal file
@@ -0,0 +1,89 @@
|
||||
-- 资产操作审计日志手工验收 SQL
|
||||
-- 表:tb_asset_operation_log
|
||||
|
||||
-- 1) 表结构与索引核验
|
||||
SELECT table_name
|
||||
FROM information_schema.tables
|
||||
WHERE table_schema = 'public'
|
||||
AND table_name = 'tb_asset_operation_log';
|
||||
|
||||
SELECT indexname, indexdef
|
||||
FROM pg_indexes
|
||||
WHERE schemaname = 'public'
|
||||
AND tablename = 'tb_asset_operation_log'
|
||||
ORDER BY indexname;
|
||||
|
||||
-- 2) 最近 50 条审计日志
|
||||
SELECT id,
|
||||
created_at,
|
||||
operator_type,
|
||||
operator_id,
|
||||
asset_type,
|
||||
asset_id,
|
||||
asset_identifier,
|
||||
operation_type,
|
||||
result_status,
|
||||
batch_total,
|
||||
success_count,
|
||||
fail_count
|
||||
FROM tb_asset_operation_log
|
||||
ORDER BY id DESC
|
||||
LIMIT 50;
|
||||
|
||||
-- 3) success / failed / denied 三态分布
|
||||
SELECT operation_type, result_status, COUNT(*) AS cnt
|
||||
FROM tb_asset_operation_log
|
||||
GROUP BY operation_type, result_status
|
||||
ORDER BY operation_type, result_status;
|
||||
|
||||
-- 4) before/after 镜像完整性核验(近 200 条)
|
||||
SELECT id,
|
||||
operation_type,
|
||||
result_status,
|
||||
(before_data IS NOT NULL) AS has_before,
|
||||
(after_data IS NOT NULL) AS has_after
|
||||
FROM tb_asset_operation_log
|
||||
ORDER BY id DESC
|
||||
LIMIT 200;
|
||||
|
||||
-- 5) 批量聚合字段核验(batch_total/success_count/fail_count)
|
||||
SELECT id,
|
||||
operation_type,
|
||||
batch_total,
|
||||
success_count,
|
||||
fail_count,
|
||||
(batch_total >= success_count + fail_count) AS counter_ok
|
||||
FROM tb_asset_operation_log
|
||||
WHERE batch_total > 0
|
||||
ORDER BY id DESC
|
||||
LIMIT 200;
|
||||
|
||||
-- 6) 系统触发语义核验
|
||||
SELECT id,
|
||||
operation_type,
|
||||
operator_type,
|
||||
operator_id,
|
||||
operator_name,
|
||||
result_status
|
||||
FROM tb_asset_operation_log
|
||||
WHERE operation_type IN ('card_auto_stop', 'card_auto_start')
|
||||
ORDER BY id DESC
|
||||
LIMIT 100;
|
||||
|
||||
-- 7) WiFi 脱敏核验(不应出现明文密码)
|
||||
SELECT id,
|
||||
operation_type,
|
||||
after_data
|
||||
FROM tb_asset_operation_log
|
||||
WHERE operation_type = 'device_set_wifi'
|
||||
ORDER BY id DESC
|
||||
LIMIT 20;
|
||||
|
||||
-- 8) 拒绝与失败热点排查
|
||||
SELECT operation_type,
|
||||
result_status,
|
||||
COUNT(*) AS cnt
|
||||
FROM tb_asset_operation_log
|
||||
WHERE result_status IN ('failed', 'denied')
|
||||
GROUP BY operation_type, result_status
|
||||
ORDER BY cnt DESC, operation_type;
|
||||
69
docs/add-asset-operation-audit-log/接口回放示例.md
Normal file
69
docs/add-asset-operation-audit-log/接口回放示例.md
Normal file
@@ -0,0 +1,69 @@
|
||||
# 资产审计日志接口回放示例
|
||||
|
||||
以下示例用于构造 success / failed / denied 三类日志数据。请替换 `<token>`、`<identifier>`、`<device_id>`、`<card_id>` 等占位符。
|
||||
|
||||
## 1. success 示例
|
||||
|
||||
### 1.1 设备限速(`device_speed_limit`)
|
||||
|
||||
```bash
|
||||
curl -X POST "http://127.0.0.1:8080/api/admin/devices/<identifier>/speed-limit" \
|
||||
-H "Authorization: Bearer <token>" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"speed_limit":1024}'
|
||||
```
|
||||
|
||||
### 1.2 资产轮询开关(`asset_polling_status`)
|
||||
|
||||
```bash
|
||||
curl -X PATCH "http://127.0.0.1:8080/api/admin/assets/<identifier>/polling-status" \
|
||||
-H "Authorization: Bearer <token>" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"enable_polling":false}'
|
||||
```
|
||||
|
||||
## 2. denied 示例
|
||||
|
||||
### 2.1 保护期内停机(`device_stop` / `card_manual_stop`)
|
||||
|
||||
```bash
|
||||
curl -X POST "http://127.0.0.1:8080/api/admin/assets/<identifier>/stop" \
|
||||
-H "Authorization: Bearer <token>"
|
||||
```
|
||||
|
||||
若命中保护期规则,应返回拒绝并写入 `result_status=denied`。
|
||||
|
||||
### 2.2 越权分配设备(`device_allocate`)
|
||||
|
||||
```bash
|
||||
curl -X POST "http://127.0.0.1:8080/api/admin/devices/allocate" \
|
||||
-H "Authorization: Bearer <agent-token>" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"target_shop_id":999999,"device_ids":[1,2,3],"remark":"审计验收"}'
|
||||
```
|
||||
|
||||
## 3. failed 示例
|
||||
|
||||
### 3.1 无效设备标识触发远程控制失败(`device_reboot`)
|
||||
|
||||
```bash
|
||||
curl -X POST "http://127.0.0.1:8080/api/admin/devices/<invalid-identifier>/reboot" \
|
||||
-H "Authorization: Bearer <token>"
|
||||
```
|
||||
|
||||
### 3.2 导入任务入队失败(`device_import_task_create` / `iot_card_import_task_create`)
|
||||
|
||||
通过临时关闭队列依赖或传入无效参数触发失败,确认写入 `result_status=failed` 和错误摘要。
|
||||
|
||||
## 4. 结果核对
|
||||
|
||||
执行完回放后,使用:
|
||||
|
||||
- `docs/add-asset-operation-audit-log/手工验收脚本.sql`
|
||||
|
||||
重点核对:
|
||||
|
||||
1. 同一操作存在 success/failed/denied
|
||||
2. `before_data` 与 `after_data` 有效
|
||||
3. WiFi 密码字段为掩码
|
||||
4. 批量字段 `batch_total/success_count/fail_count` 合理
|
||||
102
docs/add-asset-operation-audit-log/操作内容字段说明.md
Normal file
102
docs/add-asset-operation-audit-log/操作内容字段说明.md
Normal file
@@ -0,0 +1,102 @@
|
||||
# 资产审计日志操作内容字段说明
|
||||
|
||||
## 1. 查询接口
|
||||
|
||||
- 路径:`GET /api/admin/assets/:identifier/operation-logs`
|
||||
- 说明:先按资产标识符解析资产,再查询该资产的审计日志。
|
||||
|
||||
## 2. 前端应优先使用的字段
|
||||
|
||||
为避免历史数据格式差异,前端应优先使用以下三个字段:
|
||||
|
||||
1. `operation_content_before`:操作内容(变更前)
|
||||
2. `operation_content_after`:操作内容(变更后)
|
||||
3. `operation_fields_desc`:字段中文说明(`key=字段名`,`value=中文解释`)
|
||||
|
||||
`before_data/after_data` 保留为原始审计数据,建议仅作为调试或回放使用。
|
||||
|
||||
## 3. 通用字段
|
||||
|
||||
| 字段 | 说明 |
|
||||
|---|---|
|
||||
| `operator_type` | 操作人类型中文名称(如平台用户、代理账号、企业账号、个人客户、系统) |
|
||||
| `operator_type_code` | 操作人类型编码(如 `admin_user`、`agent_user`) |
|
||||
| `operator_name` | 操作人可读名称,优先返回用户名/店铺名/企业名/客户昵称 |
|
||||
| `batch_total` | 批量总数 |
|
||||
| `success_count` | 成功数量 |
|
||||
| `fail_count` | 失败数量 |
|
||||
| `failed_items` | 失败明细 |
|
||||
| `reason` | 原因说明 |
|
||||
|
||||
## 4. 常见操作字段
|
||||
|
||||
### 4.1 分配/回收
|
||||
|
||||
| 字段 | 说明 |
|
||||
|---|---|
|
||||
| `target_shop_id` / `to_shop_id` | 目标店铺ID |
|
||||
| `source_shop_id` / `from_shop_id` | 来源店铺ID |
|
||||
| `device_ids` | 设备ID列表 |
|
||||
| `card_ids` | 卡ID列表 |
|
||||
| `selection_type` | 选择方式 |
|
||||
| `new_status` | 变更后状态 |
|
||||
|
||||
### 4.2 设备绑卡/解绑
|
||||
|
||||
| 字段 | 说明 |
|
||||
|---|---|
|
||||
| `binding_id` | 绑定记录ID |
|
||||
| `slot_position` | 插槽位置 |
|
||||
| `iot_card_id` | 卡ID |
|
||||
| `iccid` | 卡ICCID |
|
||||
| `unbind` | 是否执行解绑 |
|
||||
|
||||
### 4.3 设备停复机
|
||||
|
||||
| 字段 | 说明 |
|
||||
|---|---|
|
||||
| `success_count` | 成功数量 |
|
||||
| `fail_count` | 失败数量 |
|
||||
| `skip_count` | 跳过数量 |
|
||||
| `failed_items` | 失败明细(含ICCID和原因) |
|
||||
|
||||
### 4.4 远程控制
|
||||
|
||||
| 字段 | 说明 |
|
||||
|---|---|
|
||||
| `speed_limit` | 限速值(KB/s) |
|
||||
| `ssid` | WiFi 名称 |
|
||||
| `password` | WiFi 密码(已脱敏) |
|
||||
| `enabled` | WiFi 开关状态 |
|
||||
| `target_iccid` | 目标ICCID |
|
||||
| `switch_mode` | 切卡模式(0自动/1手动) |
|
||||
|
||||
### 4.5 轮询与实名
|
||||
|
||||
| 字段 | 说明 |
|
||||
|---|---|
|
||||
| `enable_polling` | 轮询开关 |
|
||||
| `realname_policy` | 实名策略(none/before_order/after_order) |
|
||||
| `real_name_status` | 实名状态(0未实名/1已实名) |
|
||||
|
||||
## 5. 兼容性说明
|
||||
|
||||
历史日志中 `before_data` 可能是资产快照而不是操作内容。接口已做归一化处理:
|
||||
|
||||
- 自动抽取并返回 `operation_content_before/after`
|
||||
- 当历史数据缺少“变更前操作内容”时,会按“变更后字段”与资产快照尽量回填
|
||||
- 历史日志中的 `后台用户#ID`、`代理用户#ID` 等占位名称,接口会尽量按当前账号资料补全为可读名称
|
||||
|
||||
因此,前端无需再自行解析 `before_data/after_data` 的多种历史格式。
|
||||
|
||||
## 6. 前端处理建议
|
||||
|
||||
建议前端按以下优先级渲染变更明细:
|
||||
|
||||
1. 遍历 `operation_content_after` 的字段键作为“变更项集合”
|
||||
2. 字段名展示优先使用 `operation_fields_desc[key]`,没有则回退为 `key`
|
||||
3. 变更前值读取 `operation_content_before[key]`
|
||||
4. 变更后值读取 `operation_content_after[key]`
|
||||
5. 当前值为 `null` 时按“未知/未记录”展示,不要直接当空字符串
|
||||
|
||||
`before_data/after_data` 建议只用于问题排查,不参与主界面渲染。
|
||||
File diff suppressed because it is too large
Load Diff
@@ -48,7 +48,7 @@
|
||||
|
||||
### 基础说明
|
||||
|
||||
- 路径参数 `asset_type` 取值:`card`(卡)或 `device`(设备)
|
||||
- 资产相关路由已统一为 `:identifier` 口径(虚拟号 / ICCID / IMEI / SN / MSISDN)
|
||||
- 企业账号调用 `resolve` 接口会返回 403
|
||||
|
||||
---
|
||||
@@ -73,9 +73,12 @@
|
||||
| `real_name_status` | int | 实名状态:0 未实名 / 1 实名中 / 2 已实名 |
|
||||
| `network_status` | int | 网络状态:0 停机 / 1 开机(仅 card) |
|
||||
| `current_package` | string | 当前套餐名称(无则空) |
|
||||
| `package_total_mb` | int64 | 当前套餐总虚流量 MB |
|
||||
| `package_used_mb` | float64 | 已用虚流量 MB |
|
||||
| `package_remain_mb` | float64 | 剩余虚流量 MB |
|
||||
| `current_package_usage_id` | uint | 当前主套餐的套餐使用记录 ID(无主套餐时为 null) |
|
||||
| `real_total_mb` | int64 | 当前主套餐真实总量 MB |
|
||||
| `real_used_mb` | int64 | 当前主套餐真实已用量 MB |
|
||||
| `virtual_total_mb` | int64 | 当前主套餐业务停机阈值 MB |
|
||||
| `virtual_used_mb` | float64 | 当前主套餐展示已用量 MB |
|
||||
| `reduction_pct` | float64 | 展示增幅比例,公式为 `(real_total_mb / virtual_total_mb) - 1` |
|
||||
| `device_protect_status` | string | 保护期状态:`none` / `stop` / `start`(仅 device) |
|
||||
| `activated_at` | time | 激活时间 |
|
||||
| `created_at` | time | 创建时间 |
|
||||
@@ -108,7 +111,7 @@
|
||||
|
||||
---
|
||||
|
||||
### `GET /api/admin/assets/:asset_type/:id/realtime-status`
|
||||
### `GET /api/admin/assets/:identifier/realtime-status`
|
||||
|
||||
读取资产实时状态(直接读 DB/Redis,不调网关)。
|
||||
|
||||
@@ -127,7 +130,7 @@
|
||||
|
||||
---
|
||||
|
||||
### `POST /api/admin/assets/:asset_type/:id/refresh`
|
||||
### `POST /api/admin/assets/:identifier/refresh`
|
||||
|
||||
主动调网关拉取最新数据后返回,响应结构与 `realtime-status` 完全相同。
|
||||
|
||||
@@ -135,7 +138,7 @@
|
||||
|
||||
---
|
||||
|
||||
### `GET /api/admin/assets/:asset_type/:id/packages`
|
||||
### `GET /api/admin/assets/:identifier/packages`
|
||||
|
||||
查询该资产所有套餐记录,含虚流量换算字段。
|
||||
|
||||
@@ -149,12 +152,11 @@
|
||||
| `package_type` | string | `formal`(正式套餐)/ `addon`(加油包) |
|
||||
| `status` | int | 0 待生效 / 1 生效中 / 2 已用完 / 3 已过期 / 4 已失效 |
|
||||
| `status_name` | string | 状态中文名 |
|
||||
| `data_limit_mb` | int64 | 真流量总量 MB |
|
||||
| `virtual_limit_mb` | int64 | 虚流量总量 MB(已按 virtual_ratio 换算) |
|
||||
| `data_usage_mb` | int64 | 已用真流量 MB |
|
||||
| `virtual_used_mb` | float64 | 已用虚流量 MB |
|
||||
| `virtual_remain_mb` | float64 | 剩余虚流量 MB |
|
||||
| `virtual_ratio` | float64 | 虚流量比例 |
|
||||
| `real_total_mb` | int64 | 套餐真实总量 MB |
|
||||
| `real_used_mb` | int64 | 套餐真实已用量 MB |
|
||||
| `virtual_total_mb` | int64 | 套餐业务停机阈值 MB |
|
||||
| `virtual_used_mb` | float64 | 套餐展示已用量 MB |
|
||||
| `reduction_pct` | float64 | 展示增幅比例,公式为 `(real_total_mb / virtual_total_mb) - 1` |
|
||||
| `activated_at` | time | 激活时间 |
|
||||
| `expires_at` | time | 到期时间 |
|
||||
| `master_usage_id` | uint | 主套餐 ID(加油包时有值) |
|
||||
@@ -163,9 +165,9 @@
|
||||
|
||||
---
|
||||
|
||||
### `GET /api/admin/assets/:asset_type/:id/current-package`
|
||||
### `GET /api/admin/assets/:identifier/current-package`
|
||||
|
||||
查询当前生效中的主套餐,响应结构同 `packages` 数组的单项。无生效套餐时返回 404。
|
||||
查询当前生效中的主套餐,响应结构同 `packages` 数组的单项。无生效套餐时返回 `200 + null`。
|
||||
|
||||
---
|
||||
|
||||
|
||||
110
docs/traffic-model-unification/字段替换矩阵.md
Normal file
110
docs/traffic-model-unification/字段替换矩阵.md
Normal file
@@ -0,0 +1,110 @@
|
||||
# 流量模型统一字段替换矩阵
|
||||
|
||||
## 1. 目的
|
||||
|
||||
本文用于固化 `refactor-traffic-model-semantics` 变更前后的字段替换关系,作为后台 / C 端接口、OpenAPI 文档和手工回放的统一对照基线。
|
||||
|
||||
## 2. 统一后的核心字段
|
||||
|
||||
本次 contract 收口后,流量相关字段统一为:
|
||||
|
||||
| 字段 | 含义 |
|
||||
|------|------|
|
||||
| `current_package_usage_id` | 当前主套餐对应的套餐使用记录 ID,无主套餐时为 `null` |
|
||||
| `real_total_mb` | 套餐真实总量,内部来源 `tb_package_usage.data_limit_mb` |
|
||||
| `real_used_mb` | 套餐真实已用,内部来源 `tb_package_usage.data_usage_mb` |
|
||||
| `virtual_total_mb` | 套餐业务停机阈值,优先来源 `tb_package_usage.virtual_total_mb_snapshot` |
|
||||
| `virtual_used_mb` | 展示已用量,公式 `min(real_used_mb * display_gain_ratio, real_total_mb)` |
|
||||
| `reduction_pct` | 展示增幅比例,公式 `(real_total_mb / virtual_total_mb) - 1` |
|
||||
|
||||
## 3. 字段替换矩阵
|
||||
|
||||
### 3.1 后台 `GET /api/admin/assets/resolve/:identifier`
|
||||
|
||||
| 现状字段 | 现状来源 | 新字段 | 新语义 |
|
||||
|---------|---------|--------|--------|
|
||||
| `current_package` | 主套餐名称 | `current_package` | 保留,语义不变 |
|
||||
| `package_total_mb` | `data_limit_mb / virtual_ratio` | `real_total_mb` | 真总量 |
|
||||
| `package_used_mb` | `data_usage_mb / virtual_ratio` | `real_used_mb` | 真已用 |
|
||||
| `package_remain_mb` | `(data_limit_mb - data_usage_mb) / virtual_ratio` | `virtual_total_mb` / `virtual_used_mb` | 不再返回 remain,改为显式真/虚语义 |
|
||||
| 无 | 无 | `current_package_usage_id` | 当前主套餐使用记录 ID |
|
||||
| 无 | 无 | `reduction_pct` | 展示增幅比例 |
|
||||
|
||||
### 3.2 后台 `GET /api/admin/assets/:identifier/current-package`
|
||||
|
||||
| 现状字段 | 现状来源 | 新字段 | 新语义 |
|
||||
|---------|---------|--------|--------|
|
||||
| `data_limit_mb` | `tb_package_usage.data_limit_mb` | `real_total_mb` | 真总量 |
|
||||
| `data_usage_mb` | `tb_package_usage.data_usage_mb` | `real_used_mb` | 真已用 |
|
||||
| `virtual_limit_mb` | `data_limit_mb / virtual_ratio` | `virtual_total_mb` | 业务停机阈值 |
|
||||
| `virtual_used_mb` | `data_usage_mb / virtual_ratio` | `virtual_used_mb` | 改为统一公式计算 |
|
||||
| `virtual_remain_mb` | `(data_limit_mb - data_usage_mb) / virtual_ratio` | 删除 | 不再返回 |
|
||||
| `virtual_ratio` | `tb_package.virtual_ratio` | `reduction_pct` | 只保留外部增幅比例 |
|
||||
| 无 | 无 | `package_usage_id` | 保留既有字段 |
|
||||
|
||||
### 3.3 后台 `GET /api/admin/assets/:identifier/packages`
|
||||
|
||||
| 现状字段 | 现状来源 | 新字段 | 新语义 |
|
||||
|---------|---------|--------|--------|
|
||||
| `data_limit_mb` | `tb_package_usage.data_limit_mb` | `real_total_mb` | 真总量 |
|
||||
| `data_usage_mb` | `tb_package_usage.data_usage_mb` | `real_used_mb` | 真已用 |
|
||||
| `virtual_limit_mb` | `data_limit_mb / virtual_ratio` | `virtual_total_mb` | 业务停机阈值 |
|
||||
| `virtual_used_mb` | `data_usage_mb / virtual_ratio` | `virtual_used_mb` | 统一公式 |
|
||||
| `virtual_remain_mb` | `(data_limit_mb - data_usage_mb) / virtual_ratio` | 删除 | 不再返回 |
|
||||
| `virtual_ratio` | `tb_package.virtual_ratio` | `reduction_pct` | 只保留外部比例 |
|
||||
| `package_usage_id` | `tb_package_usage.id` | `package_usage_id` | 保留,作为记录主键 |
|
||||
|
||||
### 3.4 C 端 `GET /api/c/v1/asset/info`
|
||||
|
||||
| 现状字段 | 现状来源 | 新字段 | 新语义 |
|
||||
|---------|---------|--------|--------|
|
||||
| `current_package` | 主套餐名称 | `current_package` | 保留 |
|
||||
| `package_total_mb` | 来自后台资产解析结果 | `real_total_mb` | 真总量 |
|
||||
| `package_used_mb` | 来自后台资产解析结果 | `real_used_mb` | 真已用 |
|
||||
| `package_remain_mb` | 来自后台资产解析结果 | `virtual_total_mb` / `virtual_used_mb` | 不再返回 remain |
|
||||
| 无 | 无 | `current_package_usage_id` | 当前主套餐使用记录 ID |
|
||||
| 无 | 无 | `reduction_pct` | 展示增幅比例 |
|
||||
|
||||
### 3.5 C 端 `GET /api/c/v1/asset/package-history`
|
||||
|
||||
| 现状字段 | 现状来源 | 新字段 | 新语义 |
|
||||
|---------|---------|--------|--------|
|
||||
| `data_limit_mb` | `tb_package_usage.data_limit_mb` | `real_total_mb` | 真总量 |
|
||||
| `data_usage_mb` | `tb_package_usage.data_usage_mb` | `real_used_mb` | 真已用 |
|
||||
| `virtual_limit_mb` | `data_limit_mb / virtual_ratio` | `virtual_total_mb` | 业务停机阈值 |
|
||||
| `virtual_used_mb` | `data_usage_mb / virtual_ratio` | `virtual_used_mb` | 统一公式 |
|
||||
| `virtual_remain_mb` | `(data_limit_mb - data_usage_mb) / virtual_ratio` | 删除 | 不再返回 |
|
||||
| `virtual_ratio` | `tb_package.virtual_ratio` | `reduction_pct` | 只保留外部比例 |
|
||||
| `package_usage_id` | `tb_package_usage.id` | `package_usage_id` | 保留 |
|
||||
|
||||
## 4. OpenAPI 对齐基线
|
||||
|
||||
当前 `docs/admin-openapi.yaml` 仍存在以下待淘汰字段定义:
|
||||
|
||||
- `package_total_mb`
|
||||
- `package_used_mb`
|
||||
- `package_remain_mb`
|
||||
- `virtual_limit_mb`
|
||||
- `virtual_remain_mb`
|
||||
- `virtual_ratio`
|
||||
|
||||
本次变更完成后,后台 OpenAPI 需仅保留:
|
||||
|
||||
- `current_package_usage_id`
|
||||
- `real_total_mb`
|
||||
- `real_used_mb`
|
||||
- `virtual_total_mb`
|
||||
- `virtual_used_mb`
|
||||
- `reduction_pct`
|
||||
|
||||
## 5. 路由基线
|
||||
|
||||
当前实际注册路由已经统一为 `:identifier` 口径:
|
||||
|
||||
- `GET /api/admin/assets/resolve/:identifier`
|
||||
- `GET /api/admin/assets/:identifier/current-package`
|
||||
- `GET /api/admin/assets/:identifier/packages`
|
||||
- `GET /api/c/v1/asset/info`
|
||||
- `GET /api/c/v1/asset/package-history`
|
||||
|
||||
因此,本次变更仅做字段契约收口,不调整上述路由路径。
|
||||
140
docs/traffic-model-unification/手工验收脚本.sql
Normal file
140
docs/traffic-model-unification/手工验收脚本.sql
Normal file
@@ -0,0 +1,140 @@
|
||||
-- 流量模型统一改造手工验收脚本
|
||||
-- 用途:验证 package_usage 快照、统一公式和 B/C 端 contract 改造后的数据库基线。
|
||||
|
||||
-- 1. 查看 package_usage 快照字段是否存在
|
||||
SELECT
|
||||
column_name,
|
||||
data_type,
|
||||
column_default,
|
||||
is_nullable
|
||||
FROM information_schema.columns
|
||||
WHERE table_schema = 'public'
|
||||
AND table_name = 'tb_package_usage'
|
||||
AND column_name IN (
|
||||
'data_limit_mb',
|
||||
'data_usage_mb',
|
||||
'virtual_total_mb_snapshot',
|
||||
'display_gain_ratio_snapshot',
|
||||
'enable_virtual_data_snapshot'
|
||||
)
|
||||
ORDER BY column_name;
|
||||
|
||||
-- 2. 查看历史记录快照回填结果
|
||||
SELECT
|
||||
pu.id AS package_usage_id,
|
||||
pu.package_id,
|
||||
pu.data_limit_mb,
|
||||
pu.data_usage_mb,
|
||||
pu.virtual_total_mb_snapshot,
|
||||
pu.display_gain_ratio_snapshot,
|
||||
pu.enable_virtual_data_snapshot,
|
||||
pu.status,
|
||||
pu.master_usage_id,
|
||||
pu.activated_at,
|
||||
pu.expires_at
|
||||
FROM tb_package_usage pu
|
||||
ORDER BY pu.id DESC
|
||||
LIMIT 50;
|
||||
|
||||
-- 3. 统一公式核验:启用虚流量时的五字段与 reduction_pct
|
||||
SELECT
|
||||
pu.id AS package_usage_id,
|
||||
pu.data_limit_mb AS real_total_mb,
|
||||
pu.data_usage_mb AS real_used_mb,
|
||||
CASE
|
||||
WHEN pu.enable_virtual_data_snapshot AND pu.virtual_total_mb_snapshot > 0 THEN pu.virtual_total_mb_snapshot
|
||||
ELSE pu.data_limit_mb
|
||||
END AS virtual_total_mb,
|
||||
CASE
|
||||
WHEN pu.enable_virtual_data_snapshot AND pu.virtual_total_mb_snapshot > 0
|
||||
THEN LEAST(pu.data_usage_mb * pu.display_gain_ratio_snapshot, pu.data_limit_mb)
|
||||
ELSE pu.data_usage_mb
|
||||
END AS virtual_used_mb,
|
||||
CASE
|
||||
WHEN pu.enable_virtual_data_snapshot AND pu.virtual_total_mb_snapshot > 0
|
||||
THEN pu.display_gain_ratio_snapshot - 1
|
||||
ELSE 0
|
||||
END AS reduction_pct
|
||||
FROM tb_package_usage pu
|
||||
ORDER BY pu.id DESC
|
||||
LIMIT 50;
|
||||
|
||||
-- 4. 停机阈值核验:统一 effective_threshold_mb
|
||||
SELECT
|
||||
pu.id AS package_usage_id,
|
||||
pu.data_limit_mb AS real_total_mb,
|
||||
pu.data_usage_mb AS real_used_mb,
|
||||
CASE
|
||||
WHEN pu.enable_virtual_data_snapshot AND pu.virtual_total_mb_snapshot > 0 THEN pu.virtual_total_mb_snapshot
|
||||
ELSE pu.data_limit_mb
|
||||
END AS effective_threshold_mb,
|
||||
CASE
|
||||
WHEN pu.data_usage_mb >= CASE
|
||||
WHEN pu.enable_virtual_data_snapshot AND pu.virtual_total_mb_snapshot > 0 THEN pu.virtual_total_mb_snapshot
|
||||
ELSE pu.data_limit_mb
|
||||
END THEN TRUE
|
||||
ELSE FALSE
|
||||
END AS is_depleted
|
||||
FROM tb_package_usage pu
|
||||
ORDER BY pu.id DESC
|
||||
LIMIT 50;
|
||||
|
||||
-- 5. 样例一:100 / 70 / 50 -> 71.43
|
||||
WITH sample AS (
|
||||
SELECT
|
||||
100::bigint AS real_total_mb,
|
||||
50::bigint AS real_used_mb,
|
||||
70::bigint AS virtual_total_mb,
|
||||
ROUND(100::numeric / 70::numeric, 6) AS display_gain_ratio
|
||||
)
|
||||
SELECT
|
||||
real_total_mb,
|
||||
real_used_mb,
|
||||
virtual_total_mb,
|
||||
display_gain_ratio,
|
||||
ROUND(LEAST(real_used_mb * display_gain_ratio, real_total_mb)::numeric, 2) AS expected_virtual_used_mb,
|
||||
ROUND((display_gain_ratio - 1)::numeric, 6) AS expected_reduction_pct
|
||||
FROM sample;
|
||||
|
||||
-- 6. 样例二:100 / 70 / 69 -> 98.57
|
||||
WITH sample AS (
|
||||
SELECT
|
||||
100::bigint AS real_total_mb,
|
||||
69::bigint AS real_used_mb,
|
||||
70::bigint AS virtual_total_mb,
|
||||
ROUND(100::numeric / 70::numeric, 6) AS display_gain_ratio
|
||||
)
|
||||
SELECT
|
||||
real_total_mb,
|
||||
real_used_mb,
|
||||
virtual_total_mb,
|
||||
display_gain_ratio,
|
||||
ROUND(LEAST(real_used_mb * display_gain_ratio, real_total_mb)::numeric, 2) AS expected_virtual_used_mb,
|
||||
ROUND((display_gain_ratio - 1)::numeric, 6) AS expected_reduction_pct
|
||||
FROM sample;
|
||||
|
||||
-- 7. 样例三:未启用虚流量时应退化为真流量视图
|
||||
WITH sample AS (
|
||||
SELECT
|
||||
100::bigint AS real_total_mb,
|
||||
69::bigint AS real_used_mb,
|
||||
FALSE AS enable_virtual_data_snapshot,
|
||||
100::bigint AS virtual_total_mb_snapshot,
|
||||
1.0::numeric AS display_gain_ratio_snapshot
|
||||
)
|
||||
SELECT
|
||||
real_total_mb,
|
||||
real_used_mb,
|
||||
CASE
|
||||
WHEN enable_virtual_data_snapshot AND virtual_total_mb_snapshot > 0 THEN virtual_total_mb_snapshot
|
||||
ELSE real_total_mb
|
||||
END AS expected_virtual_total_mb,
|
||||
CASE
|
||||
WHEN enable_virtual_data_snapshot AND virtual_total_mb_snapshot > 0 THEN LEAST(real_used_mb * display_gain_ratio_snapshot, real_total_mb)
|
||||
ELSE real_used_mb
|
||||
END AS expected_virtual_used_mb,
|
||||
CASE
|
||||
WHEN enable_virtual_data_snapshot AND virtual_total_mb_snapshot > 0 THEN display_gain_ratio_snapshot - 1
|
||||
ELSE 0
|
||||
END AS expected_reduction_pct
|
||||
FROM sample;
|
||||
149
docs/traffic-model-unification/接口回放样例.md
Normal file
149
docs/traffic-model-unification/接口回放样例.md
Normal file
@@ -0,0 +1,149 @@
|
||||
# 流量模型统一接口回放样例
|
||||
|
||||
## 1. 目的
|
||||
|
||||
本文用于给 `refactor-traffic-model-semantics` 提供固定回放样例,确保后台 / C 端接口、OpenAPI 和数据库公式在同一口径下验收。
|
||||
|
||||
## 2. 典型样例
|
||||
|
||||
### 样例 A:100 / 70 / 50 → 71.43
|
||||
|
||||
- `real_total_mb = 100`
|
||||
- `virtual_total_mb = 70`
|
||||
- `real_used_mb = 50`
|
||||
- `display_gain_ratio = 100 / 70 = 1.428571`
|
||||
- `virtual_used_mb = min(50 * 1.428571, 100) = 71.43`
|
||||
- `reduction_pct = 1.428571 - 1 = 0.428571`
|
||||
|
||||
### 样例 B:100 / 70 / 69 → 98.57
|
||||
|
||||
- `real_total_mb = 100`
|
||||
- `virtual_total_mb = 70`
|
||||
- `real_used_mb = 69`
|
||||
- `display_gain_ratio = 100 / 70 = 1.428571`
|
||||
- `virtual_used_mb = min(69 * 1.428571, 100) = 98.57`
|
||||
- `reduction_pct = 0.428571`
|
||||
|
||||
### 样例 C:100 / 70 / 70 → 100
|
||||
|
||||
- `real_total_mb = 100`
|
||||
- `virtual_total_mb = 70`
|
||||
- `real_used_mb = 70`
|
||||
- `virtual_used_mb = min(70 * 1.428571, 100) = 100`
|
||||
- 该套餐应判定为耗尽
|
||||
|
||||
### 样例 D:未启用虚流量退化为真流量视图
|
||||
|
||||
- `real_total_mb = 100`
|
||||
- `virtual_total_mb = 100`
|
||||
- `real_used_mb = 69`
|
||||
- `virtual_used_mb = 69`
|
||||
- `reduction_pct = 0`
|
||||
|
||||
## 3. 后台接口回放
|
||||
|
||||
### 3.1 资产解析
|
||||
|
||||
```bash
|
||||
curl -X GET "http://localhost:8080/api/admin/assets/resolve/<identifier>" \
|
||||
-H "Authorization: Bearer <admin-token>"
|
||||
```
|
||||
|
||||
重点核对:
|
||||
|
||||
- `current_package_usage_id`
|
||||
- `real_total_mb`
|
||||
- `real_used_mb`
|
||||
- `virtual_total_mb`
|
||||
- `virtual_used_mb`
|
||||
- `reduction_pct`
|
||||
- 不再出现 `package_total_mb` / `package_used_mb` / `package_remain_mb`
|
||||
|
||||
### 3.2 当前主套餐
|
||||
|
||||
```bash
|
||||
curl -X GET "http://localhost:8080/api/admin/assets/<identifier>/current-package" \
|
||||
-H "Authorization: Bearer <admin-token>"
|
||||
```
|
||||
|
||||
重点核对:
|
||||
|
||||
- 有主套餐时仅返回主套餐记录
|
||||
- 无主套餐时返回 `200 + null`
|
||||
- 记录字段包含 `package_usage_id + 统一五字段 + reduction_pct`
|
||||
|
||||
### 3.3 套餐历史
|
||||
|
||||
```bash
|
||||
curl -X GET "http://localhost:8080/api/admin/assets/<identifier>/packages?page=1&page_size=20" \
|
||||
-H "Authorization: Bearer <admin-token>"
|
||||
```
|
||||
|
||||
重点核对:
|
||||
|
||||
- 列表每条记录都包含 `package_usage_id + 统一五字段 + reduction_pct`
|
||||
- 不再出现 `virtual_limit_mb` / `virtual_remain_mb` / `virtual_ratio`
|
||||
|
||||
## 4. C 端接口回放
|
||||
|
||||
### 4.1 资产信息
|
||||
|
||||
```bash
|
||||
curl -X GET "http://localhost:8080/api/c/v1/asset/info?identifier=<identifier>" \
|
||||
-H "Authorization: Bearer <client-token>"
|
||||
```
|
||||
|
||||
重点核对:
|
||||
|
||||
- 仅返回当前主套餐摘要
|
||||
- 无主套餐时五字段为 `0`,`current_package_usage_id = null`
|
||||
- 不再出现 `package_total_mb` / `package_used_mb` / `package_remain_mb`
|
||||
|
||||
### 4.2 套餐历史
|
||||
|
||||
```bash
|
||||
curl -X GET "http://localhost:8080/api/c/v1/asset/package-history?identifier=<identifier>&page=1&page_size=20" \
|
||||
-H "Authorization: Bearer <client-token>"
|
||||
```
|
||||
|
||||
重点核对:
|
||||
|
||||
- 列表项统一返回 `package_usage_id + 统一五字段 + reduction_pct`
|
||||
- `generation` 过滤仍然有效
|
||||
- 不新增 `total.virtual_used_mb` 或等价聚合字段
|
||||
|
||||
## 5. OpenAPI 核对点
|
||||
|
||||
### 5.1 后台 `docs/admin-openapi.yaml`
|
||||
|
||||
必须包含:
|
||||
|
||||
- `current_package_usage_id`
|
||||
- `real_total_mb`
|
||||
- `real_used_mb`
|
||||
- `virtual_total_mb`
|
||||
- `virtual_used_mb`
|
||||
- `reduction_pct`
|
||||
|
||||
必须移除:
|
||||
|
||||
- `package_total_mb`
|
||||
- `package_used_mb`
|
||||
- `package_remain_mb`
|
||||
- `virtual_limit_mb`
|
||||
- `virtual_remain_mb`
|
||||
- `virtual_ratio`
|
||||
|
||||
## 6. 验收结论模板
|
||||
|
||||
每次回放后,按下述模板记录:
|
||||
|
||||
```markdown
|
||||
- [ ] 样例 A(100/70/50 → 71.43)通过
|
||||
- [ ] 样例 B(100/70/69 → 98.57)通过
|
||||
- [ ] 样例 C(100/70/70 → 100,且耗尽)通过
|
||||
- [ ] 样例 D(未启用虚流量退化)通过
|
||||
- [ ] 后台 resolve/current-package/packages 不再返回旧字段
|
||||
- [ ] C 端 asset/info/package-history 不再返回旧字段
|
||||
- [ ] OpenAPI 与 DTO 完全一致
|
||||
```
|
||||
634
docs/traffic-model-unification/流量模型统一改造方案.md
Normal file
634
docs/traffic-model-unification/流量模型统一改造方案.md
Normal file
@@ -0,0 +1,634 @@
|
||||
# 流量模型统一改造方案
|
||||
|
||||
## 1. 背景
|
||||
|
||||
当前项目中的流量逻辑同时存在以下几套口径:
|
||||
|
||||
1. **卡同步口径**
|
||||
- `tb_iot_card.last_gateway_reading_mb`
|
||||
- `tb_iot_card.current_month_usage_mb`
|
||||
- `tb_iot_card.data_usage_mb`
|
||||
|
||||
2. **套餐使用口径**
|
||||
- `tb_package_usage.data_limit_mb`
|
||||
- `tb_package_usage.data_usage_mb`
|
||||
|
||||
3. **展示换算口径**
|
||||
- `tb_package.virtual_ratio`
|
||||
- B 端/C 端多个接口中对 `virtual_*` 字段的换算
|
||||
|
||||
这三套口径目前没有统一到同一业务模型上,导致以下问题:
|
||||
|
||||
1. 流量同步与流量展示不是同一套业务事实源。
|
||||
2. 停机判断仍按真总量而不是按虚阈值判断。
|
||||
3. 历史套餐展示依赖 `tb_package` 当前值,缺少使用记录快照。
|
||||
4. 资产视角接口和套餐视角接口混用,前端不知道该信哪一个字段。
|
||||
5. 已存在字段 `real_data_usage_mb`、`virtual_data_usage_mb` 没有进入主链路,语义噪音较大。
|
||||
|
||||
本方案的目标不是“小修展示口径”,而是**统一整个项目的流量业务模型**。
|
||||
|
||||
---
|
||||
|
||||
## 2. 核心结论
|
||||
|
||||
### 2.1 业务事实源统一
|
||||
|
||||
**结论:业务流量永远跟随套餐,不跟随卡。**
|
||||
|
||||
- **卡表(`tb_iot_card`)** 只负责:
|
||||
- 同步上游网关累计流量
|
||||
- 计算增量
|
||||
- 记录同步状态与同步时间
|
||||
- 排查上游异常
|
||||
|
||||
- **套餐使用表(`tb_package_usage`)** 才是:
|
||||
- 停机判断事实源
|
||||
- B 端资产流量展示事实源
|
||||
- C 端流量展示事实源
|
||||
- 套餐历史与当前套餐流量事实源
|
||||
|
||||
### 2.2 流量摘要主体统一
|
||||
|
||||
**结论:流量摘要的主体应为 `package_usage_id`,不是 `asset_id`,也不是 `package_id`。**
|
||||
|
||||
原因:
|
||||
|
||||
1. 同一个资产可能存在多条套餐使用记录(主套餐、加油包、历史套餐)。
|
||||
2. 同一个 `package_id` 会被多个资产、多次购买复用。
|
||||
3. 前端真正要看的,是某一条具体套餐使用记录当前的四个业务值。
|
||||
|
||||
因此,流量摘要应以 **`tb_package_usage.id`** 为主键进行查询和展示。
|
||||
|
||||
### 2.3 前端最终只看四个业务值
|
||||
|
||||
本次统一后的对外流量模型只保留四个业务值:
|
||||
|
||||
1. `real_total_mb`:真总量
|
||||
2. `real_used_mb`:真已用
|
||||
3. `virtual_total_mb`:虚总量(原始虚阈值)
|
||||
4. `virtual_used_mb`:按比例换算后、与真总量同尺度的展示值
|
||||
|
||||
说明:
|
||||
|
||||
- 用户已确认:`virtual_total_mb` 使用**原始虚总量**,不是换算后的总量。
|
||||
- 用户已确认:`virtual_used_mb` 使用**按比例换算后的展示值**。
|
||||
|
||||
---
|
||||
|
||||
## 3. 统一后的业务定义
|
||||
|
||||
### 3.1 套餐模板层(`tb_package`)
|
||||
|
||||
套餐模板保留以下字段定义:
|
||||
|
||||
- `real_data_mb`:套餐真总量
|
||||
- `virtual_data_mb`:套餐虚总量 / 虚阈值
|
||||
- `enable_virtual_data`:是否启用虚流量
|
||||
- `virtual_ratio`:内部展示倍率,公式为 `real_data_mb / virtual_data_mb`
|
||||
|
||||
### 3.2 套餐使用层(`tb_package_usage`)
|
||||
|
||||
套餐使用记录作为最终业务事实源,统一定义为:
|
||||
|
||||
- `data_limit_mb`:**真总量快照**
|
||||
- `data_usage_mb`:**真已用**
|
||||
- `virtual_total_mb_snapshot`:**虚总量快照**
|
||||
- `display_gain_ratio_snapshot`:**展示倍率快照**
|
||||
- `enable_virtual_data_snapshot`:**是否启用虚流量快照**
|
||||
|
||||
### 3.3 四个业务值公式
|
||||
|
||||
对于任意一条 `package_usage`,统一用以下公式计算:
|
||||
|
||||
#### 场景 A:启用虚流量且 `virtual_total_mb_snapshot > 0`
|
||||
|
||||
- `real_total_mb = data_limit_mb`
|
||||
- `real_used_mb = data_usage_mb`
|
||||
- `virtual_total_mb = virtual_total_mb_snapshot`
|
||||
- `virtual_used_mb = min(real_used_mb * display_gain_ratio_snapshot, real_total_mb)`
|
||||
- `reduction_pct = display_gain_ratio_snapshot - 1`
|
||||
|
||||
#### 场景 B:未启用虚流量
|
||||
|
||||
- `real_total_mb = data_limit_mb`
|
||||
- `real_used_mb = data_usage_mb`
|
||||
- `virtual_total_mb = data_limit_mb`
|
||||
- `virtual_used_mb = real_used_mb`
|
||||
|
||||
说明:
|
||||
|
||||
1. 未启用虚流量时,虚流量视图退化为真流量视图,保证前端仍然能稳定收到四个值。
|
||||
2. `virtual_used_mb` 的上限必须封顶到 `real_total_mb`,避免出现展示值超过真总量。
|
||||
|
||||
### 3.4 停机判断公式
|
||||
|
||||
统一后的停机判断必须基于套餐使用记录:
|
||||
|
||||
#### 场景 A:启用虚流量且 `virtual_total_mb_snapshot > 0`
|
||||
|
||||
- 当 `real_used_mb >= virtual_total_mb` 时,套餐视为已耗尽
|
||||
|
||||
#### 场景 B:未启用虚流量
|
||||
|
||||
- 当 `real_used_mb >= real_total_mb` 时,套餐视为已耗尽
|
||||
|
||||
这一定义与本次 OpenSpec 的典型样例一致:
|
||||
|
||||
- 真 `100` / 虚 `70`:真已用到 `70` 时停机,展示 `virtual_used_mb = 100`
|
||||
- 未启用虚流量:真已用到 `100` 时停机,展示 `virtual_used_mb = real_used_mb`
|
||||
|
||||
### 3.5 `virtual_data_mb` 校验规则
|
||||
|
||||
本次统一后的规则为:
|
||||
|
||||
1. `enable_virtual_data = true` 时,`virtual_data_mb` 必须大于 `0`
|
||||
2. `enable_virtual_data = true` 时,`virtual_data_mb` 必须小于等于 `real_data_mb`
|
||||
3. 未启用虚流量时,展示视图退化为真流量视图
|
||||
|
||||
---
|
||||
|
||||
## 4. 同步链路统一方案
|
||||
|
||||
### 4.1 卡同步链路保留,但降级为“同步事实链路”
|
||||
|
||||
当前同步链路的核心算法可以保留:
|
||||
|
||||
1. 从网关读取当前累计流量读数 `Used`
|
||||
2. 使用 `Used - last_gateway_reading_mb` 计算本次增量
|
||||
3. 遇到上游重置窗口时,将当前读数视为新周期增量
|
||||
|
||||
该链路应继续写入卡表:
|
||||
|
||||
- `tb_iot_card.last_gateway_reading_mb`
|
||||
- `tb_iot_card.current_month_usage_mb`
|
||||
- `tb_iot_card.current_month_start_date`
|
||||
- `tb_iot_card.last_month_total_mb`
|
||||
- `tb_iot_card.last_sync_time`
|
||||
- `tb_iot_card.last_data_check_at`
|
||||
|
||||
### 4.2 卡同步增量如何进入套餐层
|
||||
|
||||
同步出的真流量增量必须继续扣减到 `tb_package_usage.data_usage_mb`。
|
||||
|
||||
统一口径:
|
||||
|
||||
- **同步层永远只同步“真流量增量”**
|
||||
- **套餐层永远只累加“真已用”**
|
||||
- **虚流量不参与同步,只参与展示换算和停机阈值判断**
|
||||
|
||||
### 4.3 `tb_iot_card.data_usage_mb` 的处理
|
||||
|
||||
当前 `tb_iot_card.data_usage_mb` 表示卡生命周期累计流量,但它不是业务展示主来源。
|
||||
|
||||
本次建议:
|
||||
|
||||
- **保留字段**
|
||||
- **降级为同步辅助/运维诊断字段**
|
||||
- 不再作为 B 端/C 端业务展示的主流量来源
|
||||
|
||||
说明:
|
||||
|
||||
1. 当前该字段写入时直接使用 `int64(increment)`,存在小数截断。
|
||||
2. 套餐层已通过 `UsageService` 的余量缓存机制处理小数累计,更适合作为业务事实源。
|
||||
|
||||
---
|
||||
|
||||
## 5. 展示链路统一方案
|
||||
|
||||
### 5.1 资产视角不再直接承载流量摘要
|
||||
|
||||
`asset` 本身不是流量摘要的正确主体,因为一个资产可能有多条套餐使用记录。
|
||||
|
||||
因此:
|
||||
|
||||
- `资产详情`
|
||||
- `资产解析`
|
||||
- `C 端资产信息`
|
||||
|
||||
不应再把流量摘要作为“资产级别唯一摘要”长期保留。
|
||||
|
||||
### 5.2 套餐视角承载流量摘要
|
||||
|
||||
流量摘要应绑定到具体的 `package_usage_id`:
|
||||
|
||||
- B 端:资产套餐列表 / 当前套餐
|
||||
- C 端:当前套餐 / 套餐历史 / 套餐流量详情
|
||||
|
||||
都应围绕 `package_usage_id` 返回四个业务值。
|
||||
|
||||
### 5.3 建议的接口模型
|
||||
|
||||
#### 方案 A:现有套餐接口直接返回四值
|
||||
|
||||
- `GET /api/admin/assets/:identifier/packages`
|
||||
- `GET /api/admin/assets/:identifier/current-package`
|
||||
- `GET /api/c/v1/asset/package-history`
|
||||
|
||||
每条 `AssetPackageResponse` 直接返回:
|
||||
|
||||
- `real_total_mb`
|
||||
- `real_used_mb`
|
||||
- `virtual_total_mb`
|
||||
- `virtual_used_mb`
|
||||
|
||||
#### 方案 B:新增“套餐流量摘要”接口
|
||||
|
||||
按 `package_usage_id` 查询单条套餐流量摘要,例如:
|
||||
|
||||
- `GET /api/admin/asset-packages/:package_usage_id/traffic-summary`
|
||||
- `GET /api/c/v1/asset/package-usages/:package_usage_id/traffic-summary`
|
||||
|
||||
推荐做法:
|
||||
|
||||
1. 套餐列表接口仍返回四值,方便列表页直接展示
|
||||
2. 同时提供单条摘要接口,供详情弹窗/详情页按 `package_usage_id` 精确查询
|
||||
|
||||
### 5.4 本次建议下线的旧展示字段
|
||||
|
||||
以下字段语义模糊,应从对外 DTO 中逐步移除:
|
||||
|
||||
- `package_total_mb`
|
||||
- `package_used_mb`
|
||||
- `package_remain_mb`
|
||||
- `virtual_remain_mb`
|
||||
|
||||
替代为显式字段:
|
||||
|
||||
- `real_total_mb`
|
||||
- `real_used_mb`
|
||||
- `virtual_total_mb`
|
||||
- `virtual_used_mb`
|
||||
|
||||
---
|
||||
|
||||
## 6. 数据库改造方案
|
||||
|
||||
### 6.1 本次必须新增的字段
|
||||
|
||||
在 `tb_package_usage` 新增:
|
||||
|
||||
1. `virtual_total_mb_snapshot BIGINT NOT NULL DEFAULT 0`
|
||||
2. `display_gain_ratio_snapshot DECIMAL(18,6) NOT NULL DEFAULT 1.0`
|
||||
3. `enable_virtual_data_snapshot BOOLEAN NOT NULL DEFAULT FALSE`
|
||||
|
||||
### 6.2 为什么必须加快照字段
|
||||
|
||||
如果不加快照,会有三个问题:
|
||||
|
||||
1. 历史套餐展示依赖 `tb_package` 当前值,模板一改,历史流量就漂
|
||||
2. 历史停机判断无法复现当时口径
|
||||
3. 退款、历史订单、历史套餐详情都无法稳定回放
|
||||
|
||||
### 6.3 回填策略
|
||||
|
||||
对已有 `tb_package_usage` 数据执行回填:
|
||||
|
||||
#### 若 `tb_package.enable_virtual_data = true` 且 `tb_package.virtual_data_mb > 0`
|
||||
|
||||
- `virtual_total_mb_snapshot = tb_package.virtual_data_mb`
|
||||
- `display_gain_ratio_snapshot = tb_package.virtual_ratio`
|
||||
- `enable_virtual_data_snapshot = true`
|
||||
|
||||
#### 否则
|
||||
|
||||
- `virtual_total_mb_snapshot = data_limit_mb`
|
||||
- `display_gain_ratio_snapshot = 1.0`
|
||||
- `enable_virtual_data_snapshot = false`
|
||||
|
||||
### 6.4 现有字段暂不重命名
|
||||
|
||||
本次不建议直接重命名以下数据库字段:
|
||||
|
||||
- `tb_package_usage.data_limit_mb`
|
||||
- `tb_package_usage.data_usage_mb`
|
||||
|
||||
原因:
|
||||
|
||||
1. 改名会牵涉大量 SQL、模型、历史脚本、迁移与兼容问题
|
||||
2. 当前可以通过“统一文义 + 新增快照字段”先完成业务收口
|
||||
|
||||
本次统一后的文义为:
|
||||
|
||||
- `data_limit_mb` = 真总量
|
||||
- `data_usage_mb` = 真已用
|
||||
|
||||
### 6.5 最终可删除字段
|
||||
|
||||
以下字段当前未进入主链路,本轮已在迁移中物理删除:
|
||||
|
||||
- `tb_package_usage.real_data_usage_mb`
|
||||
- `tb_package_usage.virtual_data_usage_mb`
|
||||
|
||||
执行顺序:
|
||||
|
||||
1. 代码层先完全停止依赖它们
|
||||
2. 再通过独立迁移删除数据库旧列
|
||||
|
||||
---
|
||||
|
||||
## 7. 代码改造清单
|
||||
|
||||
### 7.1 数据模型与迁移
|
||||
|
||||
需要改动:
|
||||
|
||||
- `migrations/`:新增 `tb_package_usage` 快照字段迁移与历史回填
|
||||
- `internal/model/package.go`
|
||||
|
||||
改动内容:
|
||||
|
||||
1. `PackageUsage` 增加三类快照字段
|
||||
2. 为 `data_limit_mb`、`data_usage_mb` 的注释补充“真总量 / 真已用”的语义说明
|
||||
3. 为后续删除 `real_data_usage_mb`、`virtual_data_usage_mb` 做准备
|
||||
|
||||
### 7.2 套餐创建与激活链路
|
||||
|
||||
需要改动:
|
||||
|
||||
- `internal/service/order/service.go`
|
||||
- `internal/task/auto_purchase.go`
|
||||
|
||||
改动内容:
|
||||
|
||||
1. 创建 `PackageUsage` 时写入:
|
||||
- `data_limit_mb = pkg.RealDataMB`
|
||||
- `virtual_total_mb_snapshot`
|
||||
- `display_gain_ratio_snapshot`
|
||||
- `enable_virtual_data_snapshot`
|
||||
2. 所有主套餐、加油包、自动购包路径保持一致
|
||||
|
||||
### 7.3 套餐配置校验
|
||||
|
||||
需要改动:
|
||||
|
||||
- `internal/service/package/service.go`
|
||||
|
||||
改动内容:
|
||||
|
||||
1. 保留“启用虚流量时必须填写虚流量”的校验
|
||||
2. 收紧为 `virtual_data_mb > 0 且 <= real_data_mb`
|
||||
3. `virtual_ratio` 继续使用 `real_data_mb / virtual_data_mb`
|
||||
|
||||
### 7.4 流量同步链路
|
||||
|
||||
需要改动:
|
||||
|
||||
- `internal/task/polling_carddata_handler.go`
|
||||
- `internal/service/iot_card/service.go`
|
||||
|
||||
改动内容:
|
||||
|
||||
1. 核心同步算法保持不变
|
||||
2. 明确注释与日志:该链路同步的是“真流量增量”
|
||||
3. 保持将真增量扣减到套餐层
|
||||
4. 不在该链路引入任何虚流量换算逻辑
|
||||
|
||||
### 7.5 套餐扣减链路
|
||||
|
||||
需要改动:
|
||||
|
||||
- `internal/service/package/usage_service.go`
|
||||
|
||||
改动内容:
|
||||
|
||||
1. `data_usage_mb` 继续作为真已用累加
|
||||
2. 套餐是否“已用完”必须改为按“虚阈值快照或真总量”判断
|
||||
3. 加油包 / 主套餐的耗尽逻辑全部统一到快照字段
|
||||
4. 日记录 `PackageUsageDailyRecord` 继续记录真流量使用量
|
||||
|
||||
### 7.6 停机判断链路
|
||||
|
||||
需要改动:
|
||||
|
||||
- `internal/service/iot_card/stop_resume_service.go`
|
||||
|
||||
改动内容:
|
||||
|
||||
1. `isTrafficExhausted` 改为基于 `package_usage` 快照字段判断
|
||||
2. 启用虚流量时,按 `real_used >= virtual_total_snapshot`
|
||||
3. 未启用时,按 `real_used >= real_total`
|
||||
|
||||
### 7.7 套餐重置链路
|
||||
|
||||
需要改动:
|
||||
|
||||
- `internal/service/package/reset_service.go`
|
||||
- `internal/store/postgres/package_usage_store.go`
|
||||
|
||||
改动内容:
|
||||
|
||||
1. 重置时继续清零 `data_usage_mb`
|
||||
2. 重置后根据快照阈值恢复 `status=active`
|
||||
3. 不引入额外的虚流量计数器
|
||||
|
||||
### 7.8 后台管理接口
|
||||
|
||||
需要改动:
|
||||
|
||||
- `internal/service/asset/service.go`
|
||||
- `internal/model/dto/asset_dto.go`
|
||||
- 如有前端依赖生成文档,还需同步更新 `docs/admin-openapi.yaml`
|
||||
|
||||
改动内容:
|
||||
|
||||
1. `AssetResolveResponse` 去掉资产级别模糊流量摘要字段
|
||||
2. `AssetPackageResponse` 改为输出四个显式业务值:
|
||||
- `real_total_mb`
|
||||
- `real_used_mb`
|
||||
- `virtual_total_mb`
|
||||
- `virtual_used_mb`
|
||||
3. `packages` / `current-package` 统一按 `package_usage_id` 展示和计算
|
||||
|
||||
### 7.9 C 端接口
|
||||
|
||||
需要改动:
|
||||
|
||||
- `internal/handler/app/client_asset.go`
|
||||
- `internal/model/dto/client_asset_dto.go`
|
||||
- `internal/service/package/customer_view_service.go`
|
||||
|
||||
改动内容:
|
||||
|
||||
1. C 端资产信息页不再把资产本身当作唯一流量摘要主体
|
||||
2. C 端套餐历史/当前套餐改为套餐视角四值
|
||||
3. 如果需要单条摘要页,则新增按 `package_usage_id` 查询的接口
|
||||
|
||||
### 7.10 套餐商品展示联动确认
|
||||
|
||||
需要关注:
|
||||
|
||||
- `internal/handler/app/client_asset.go` 中可购套餐列表当前 `data_allowance` 在启用虚流量时直接展示 `virtual_data_mb`
|
||||
|
||||
该处不是本次核心链路,但建议联动确认:
|
||||
|
||||
1. 商品展示要展示真流量、虚流量,还是只展示一种
|
||||
2. 若前端也要透明展示两种总量,DTO 需补充显式字段
|
||||
|
||||
---
|
||||
|
||||
## 8. DTO 与接口建议
|
||||
|
||||
### 8.1 建议新增或替换的字段
|
||||
|
||||
建议在套餐相关 DTO 中统一使用:
|
||||
|
||||
- `real_total_mb`
|
||||
- `real_used_mb`
|
||||
- `virtual_total_mb`
|
||||
- `virtual_used_mb`
|
||||
|
||||
### 8.2 建议废弃的字段
|
||||
|
||||
建议废弃:
|
||||
|
||||
- `package_total_mb`
|
||||
- `package_used_mb`
|
||||
- `package_remain_mb`
|
||||
- `virtual_limit_mb`
|
||||
- `virtual_remain_mb`
|
||||
- `data_limit_mb`(对外 DTO 层)
|
||||
- `data_usage_mb`(对外 DTO 层)
|
||||
|
||||
说明:
|
||||
|
||||
数据库内部暂时仍可保留 `data_limit_mb` / `data_usage_mb`,但 DTO 层应改为语义明确的新字段。
|
||||
|
||||
### 8.3 接口职责建议
|
||||
|
||||
#### 卡实时状态接口
|
||||
|
||||
保留同步诊断字段,例如:
|
||||
|
||||
- `last_gateway_reading_mb`
|
||||
- `current_month_usage_mb`
|
||||
- `last_sync_time`
|
||||
|
||||
#### 套餐流量接口
|
||||
|
||||
只返回四个业务值,不再掺杂卡级同步字段。
|
||||
|
||||
---
|
||||
|
||||
## 9. 迁移与上线顺序
|
||||
|
||||
建议按以下顺序执行:
|
||||
|
||||
### 第一步:数据库准备
|
||||
|
||||
1. 新增 `tb_package_usage` 快照字段
|
||||
2. 回填历史数据
|
||||
3. 为后续删列保留可回滚窗口
|
||||
|
||||
### 第二步:写链路切换
|
||||
|
||||
1. 套餐激活/自动购包路径写入快照字段
|
||||
2. 扣减逻辑与停机逻辑改为读取快照字段
|
||||
|
||||
### 第三步:读链路切换
|
||||
|
||||
1. B 端套餐接口改为返回四个业务值
|
||||
2. C 端套餐接口改为返回四个业务值
|
||||
3. 资产级模糊流量摘要降级或移除
|
||||
|
||||
### 第四步:前端联调
|
||||
|
||||
1. 管理后台改为只消费套餐视角四值
|
||||
2. C 端改为只消费套餐视角四值
|
||||
3. 如需套餐流量详情页,按 `package_usage_id` 查询
|
||||
|
||||
### 第五步:清理旧字段
|
||||
|
||||
1. 停止使用 `real_data_usage_mb` / `virtual_data_usage_mb`
|
||||
2. 物理删除旧列
|
||||
3. 对外移除旧 DTO 字段
|
||||
|
||||
---
|
||||
|
||||
## 10. 手工验证清单
|
||||
|
||||
本项目不写自动化测试,本次改造建议用以下方式手工验证:
|
||||
|
||||
### 10.1 PostgreSQL 验证
|
||||
|
||||
验证项:
|
||||
|
||||
1. `tb_package_usage` 快照字段是否已正确回填
|
||||
2. 启用虚流量的套餐是否满足:
|
||||
- `virtual_total_mb_snapshot = tb_package.virtual_data_mb`
|
||||
- `display_gain_ratio_snapshot = tb_package.virtual_ratio`
|
||||
3. 未启用虚流量的套餐是否满足:
|
||||
- `virtual_total_mb_snapshot = data_limit_mb`
|
||||
- `display_gain_ratio_snapshot = 1.0`
|
||||
|
||||
### 10.2 上游同步验证
|
||||
|
||||
验证项:
|
||||
|
||||
1. 轮询后 `tb_iot_card.last_gateway_reading_mb` 是否更新
|
||||
2. 同步增量是否正确进入 `tb_package_usage.data_usage_mb`
|
||||
3. 跨月与运营商重置窗口时增量是否正确
|
||||
|
||||
### 10.3 停机验证
|
||||
|
||||
验证项:
|
||||
|
||||
1. 真 `100` / 虚 `70`:真已用到 `70` 时停机
|
||||
2. 真 `100` / 虚 `70` / 真已用 `69`:展示 `virtual_used_mb = 98.57`
|
||||
3. 未启用虚流量:真已用到真总量时停机
|
||||
|
||||
### 10.4 展示验证
|
||||
|
||||
验证项:
|
||||
|
||||
1. B 端套餐列表与当前套餐返回四个业务值
|
||||
2. C 端套餐信息返回四个业务值
|
||||
3. 资产详情页不再混用卡同步流量与套餐业务流量
|
||||
|
||||
---
|
||||
|
||||
## 11. 本次改造范围边界
|
||||
|
||||
### 本次必须做
|
||||
|
||||
1. 套餐使用记录补快照
|
||||
2. 同步、扣减、停机、展示统一到套餐层
|
||||
3. DTO 和接口统一四个业务值
|
||||
4. 资产流量视角切换为套餐视角
|
||||
|
||||
### 本次不建议强行做
|
||||
|
||||
1. 直接重命名数据库旧字段
|
||||
2. 在本轮立即物理删除所有历史字段
|
||||
3. 把卡同步辅助字段全部移除
|
||||
|
||||
原因:
|
||||
|
||||
1. 本次首要目标是统一业务口径,先消除逻辑混乱
|
||||
2. 物理清理应放在口径稳定后单独做
|
||||
|
||||
---
|
||||
|
||||
## 12. 最终结论
|
||||
|
||||
本次流量统一改造的核心原则只有两条:
|
||||
|
||||
1. **同步看卡,业务看套餐**
|
||||
2. **展示永远以 `package_usage_id` 为主体,不以资产为主体**
|
||||
|
||||
按本方案落地后:
|
||||
|
||||
1. 同步链路只负责真流量增量采集
|
||||
2. 套餐链路负责真已用、停机阈值和四值展示
|
||||
3. B 端和 C 端最终只消费:
|
||||
- `real_total_mb`
|
||||
- `real_used_mb`
|
||||
- `virtual_total_mb`
|
||||
- `virtual_used_mb`
|
||||
|
||||
这套模型可以同时覆盖:
|
||||
|
||||
1. 真总量大于虚阈值
|
||||
2. 真总量小于虚阈值
|
||||
3. 未启用虚流量
|
||||
4. 历史套餐快照展示
|
||||
5. 当前套餐与套餐历史流量统一展示
|
||||
103
docs/unified-export-task-system/功能总结.md
Normal file
103
docs/unified-export-task-system/功能总结.md
Normal file
@@ -0,0 +1,103 @@
|
||||
# 统一导出任务功能总结
|
||||
|
||||
## 1. 功能概览
|
||||
|
||||
本次交付新增统一导出任务系统,提供全局任务入口,支持:
|
||||
|
||||
- 创建导出任务(`scene=device/iot_card`,`format=xlsx/csv`)
|
||||
- 导出任务列表与详情
|
||||
- 导出任务取消(待处理/处理中)
|
||||
- 异步三段式执行:`dispatch -> shard -> finalize`
|
||||
- 分片产物与最终产物上传 OSS
|
||||
- 详情接口直出下载链接(固定 24 小时)
|
||||
|
||||
## 2. 新增接口
|
||||
|
||||
- `POST /api/admin/export-tasks`:创建导出任务
|
||||
- `GET /api/admin/export-tasks`:导出任务列表(支持 `scene/status/time` 过滤)
|
||||
- `GET /api/admin/export-tasks/:id`:导出任务详情(已完成时返回 `download_url`)
|
||||
- `POST /api/admin/export-tasks/:id/cancel`:取消导出任务
|
||||
|
||||
## 3. 状态机
|
||||
|
||||
### 3.1 主任务状态
|
||||
|
||||
- `1` 待处理
|
||||
- `2` 处理中
|
||||
- `3` 已完成
|
||||
- `4` 已失败
|
||||
- `5` 已取消
|
||||
|
||||
### 3.2 分片状态
|
||||
|
||||
- `1` 待处理
|
||||
- `2` 处理中
|
||||
- `3` 已成功
|
||||
- `4` 已失败
|
||||
- `5` 已取消
|
||||
|
||||
## 4. 核心设计
|
||||
|
||||
### 4.1 模型与迁移
|
||||
|
||||
新增两张表:
|
||||
|
||||
- `tb_export_task`:主任务
|
||||
- `tb_export_shard_task`:分片任务
|
||||
|
||||
关键点:
|
||||
|
||||
- 无外键,仅使用 `task_id` 做逻辑关联
|
||||
- 主任务保留进度、分片统计、权限快照、产物信息
|
||||
- 分片任务保留游标范围、分片产物、执行状态
|
||||
|
||||
### 4.2 场景策略注册中心
|
||||
|
||||
新增 `internal/exporter` 场景注册中心与策略接口,首批接入:
|
||||
|
||||
- `device` 场景策略
|
||||
- `iot_card` 场景策略
|
||||
|
||||
统一定义:
|
||||
|
||||
- `Headers()`:导出表头
|
||||
- `NextShardBoundary()`:按 keyset 方式计算分片边界
|
||||
- `QueryRows()`:按分片边界查询行数据
|
||||
|
||||
### 4.3 执行链路
|
||||
|
||||
- `export:dispatch`:构建分片并入队 `export:shard`
|
||||
- `export:shard`:按分片查询数据、写文件、上传 OSS、回写分片结果
|
||||
- `export:finalize`:汇总分片结果并生成最终文件,更新主任务状态
|
||||
|
||||
### 4.4 幂等与重试安全
|
||||
|
||||
- dispatch/shard/finalize 增加 Redis 锁,避免并发重复执行
|
||||
- 分片状态条件更新,重复消费不会重复计数
|
||||
- shard/finalize 利用 Asynq 重试上下文,在最终重试失败时才落库失败状态
|
||||
|
||||
### 4.5 取消语义
|
||||
|
||||
- 待处理:直接更新为已取消
|
||||
- 处理中:仅设置 `cancel_requested=true`,由 Worker 检查点协作停止
|
||||
- dispatch/shard/finalize 均增加取消检查点
|
||||
|
||||
### 4.6 下载链接
|
||||
|
||||
- 仅 `status=已完成` 返回 `download_url`
|
||||
- 链接有效期固定 `24h`(常量 `ExportDownloadURLExpire`)
|
||||
- URL 由详情接口实时生成,过期后再次访问可刷新
|
||||
|
||||
## 5. 权限与可见性
|
||||
|
||||
- 平台/超管:可见全部导出任务
|
||||
- 代理:仅可见其店铺范围内创建的导出任务
|
||||
- 详情越权统一返回:`无权限操作该资源或资源不存在`
|
||||
|
||||
## 6. 文档与生成器更新
|
||||
|
||||
已同步更新:
|
||||
|
||||
- 路由文档(新增 `导出任务` Tag 的 4 个接口)
|
||||
- 文档构建 Handler 集合(`pkg/openapi/handlers.go` 新增 `ExportTask`)
|
||||
- OpenAPI 文件重新生成:`docs/admin-openapi.yaml`
|
||||
56
docs/unified-export-task-system/手工验证清单.md
Normal file
56
docs/unified-export-task-system/手工验证清单.md
Normal file
@@ -0,0 +1,56 @@
|
||||
# 统一导出任务手工验证清单
|
||||
|
||||
## 验证工具与原则
|
||||
|
||||
- 数据库结构与数据状态:使用 PostgreSQL MCP(只读)
|
||||
- 接口契约验证:使用 Postman 或 curl
|
||||
- 明确约束:不新增自动化测试文件(`*_test.go`)
|
||||
|
||||
## A. 创建导出任务
|
||||
|
||||
- [ ] `POST /api/admin/export-tasks`,`scene=device`,`format=xlsx`
|
||||
- [ ] `POST /api/admin/export-tasks`,`scene=iot_card`,`format=csv`
|
||||
- [ ] 非法 `scene`(如 `order`)返回参数错误
|
||||
- [ ] 非法 `format`(如 `pdf`)返回参数错误
|
||||
|
||||
## B. 列表与过滤
|
||||
|
||||
- [ ] `GET /api/admin/export-tasks?page=1&page_size=20` 默认分页生效
|
||||
- [ ] `scene` 过滤生效(`device` / `iot_card`)
|
||||
- [ ] `status` 过滤生效(1~5)
|
||||
- [ ] `start_time/end_time` 时间过滤生效
|
||||
- [ ] `page_size > 100` 时被限制到 100
|
||||
|
||||
## C. 详情与下载
|
||||
|
||||
- [ ] 未完成任务详情不返回 `download_url`
|
||||
- [ ] 已完成任务详情返回 `download_url`
|
||||
- [ ] `download_url` 有效期固定 24 小时
|
||||
- [ ] 过期后再次访问详情可刷新新链接
|
||||
|
||||
## D. 取消语义
|
||||
|
||||
- [ ] 待处理任务可取消,状态立即变为已取消
|
||||
- [ ] 处理中任务可提交取消请求(`cancel_requested=true`)
|
||||
- [ ] 已完成/已失败/已取消任务再次取消被拒绝
|
||||
- [ ] 重复取消处理中任务幂等返回
|
||||
|
||||
## E. 异步链路(dispatch -> shard -> finalize)
|
||||
|
||||
- [ ] 创建任务后 `export:dispatch` 被消费
|
||||
- [ ] 分片任务写入 `tb_export_shard_task`
|
||||
- [ ] `export:shard` 执行后分片状态更新,产物写入 OSS
|
||||
- [ ] `export:finalize` 汇总后主任务状态收敛
|
||||
- [ ] 分片失败时主任务最终失败,错误信息可追踪
|
||||
|
||||
## F. 数据权限与可见性
|
||||
|
||||
- [ ] 平台用户可见全量任务
|
||||
- [ ] 代理仅可见其店铺范围任务
|
||||
- [ ] 越权访问详情统一返回:`无权限操作该资源或资源不存在`
|
||||
|
||||
## G. 数据库核查项(PostgreSQL MCP)
|
||||
|
||||
- [ ] `tb_export_task` 表存在,字段与注释完整
|
||||
- [ ] `tb_export_shard_task` 表存在,字段与注释完整
|
||||
- [ ] 关键索引存在并生效(任务号唯一索引、`task_id+status`、`scene+status+created_at`)
|
||||
59
docs/unified-export-task-system/验收记录.md
Normal file
59
docs/unified-export-task-system/验收记录.md
Normal file
@@ -0,0 +1,59 @@
|
||||
# 统一导出任务验收记录
|
||||
|
||||
## 1. 验收范围
|
||||
|
||||
- 代码编译验证
|
||||
- 数据库迁移验证(PostgreSQL MCP)
|
||||
- OpenAPI 生成验证
|
||||
- 手工接口链路验证准备(受环境变量约束)
|
||||
|
||||
## 2. 已完成结果
|
||||
|
||||
### 2.1 编译结果
|
||||
|
||||
- 命令:`go build ./...`
|
||||
- 结果:通过
|
||||
|
||||
### 2.2 OpenAPI 结果
|
||||
|
||||
- 命令:`go run cmd/gendocs/main.go`
|
||||
- 结果:通过
|
||||
- 产物:`docs/admin-openapi.yaml`
|
||||
|
||||
### 2.3 数据库迁移结果
|
||||
|
||||
- 命令:`migrate -path migrations -database <...> up 1`
|
||||
- 结果:`135/u create_export_task_tables`
|
||||
|
||||
### 2.4 PostgreSQL MCP 结构校验
|
||||
|
||||
已确认:
|
||||
|
||||
- 存在 `tb_export_task`
|
||||
- 存在 `tb_export_shard_task`
|
||||
- 表注释和字段注释已生效
|
||||
|
||||
已确认索引:
|
||||
|
||||
- `idx_export_task_no`(唯一,软删条件)
|
||||
- `idx_export_task_scene_status_created_at`
|
||||
- `idx_export_task_creator_user_id`
|
||||
- `idx_export_task_creator_shop_id`
|
||||
- `idx_export_task_creator_enterprise_id`
|
||||
- `idx_export_shard_task_task_shard_no`(唯一,软删条件)
|
||||
- `idx_export_shard_task_task_status`
|
||||
|
||||
## 3. 待手工执行项(环境阻塞)
|
||||
|
||||
以下项已准备 curl/Postman 清单与模板,但当前环境缺少 API 启动所需配置(`JUNHONG_REDIS_ADDRESS`、`JUNHONG_JWT_SECRET_KEY` 等),暂未执行:
|
||||
|
||||
- 接口契约验证(创建/列表/详情/取消)
|
||||
- 异步状态流转观察(dispatch/shard/finalize)
|
||||
- 两个场景(`device` / `iot_card`)各 `xlsx` 与 `csv` 完整导出
|
||||
- 取消场景回归(待处理取消、处理中取消、重复取消)
|
||||
|
||||
可复现命令与模板见:
|
||||
|
||||
- `docs/unified-export-task-system/手工验证清单.md`
|
||||
- `docs/unified-export-task-system/验收记录模板.md`
|
||||
- `docs/unified-export-task-system/验证数据样本.md`
|
||||
64
docs/unified-export-task-system/验收记录模板.md
Normal file
64
docs/unified-export-task-system/验收记录模板.md
Normal file
@@ -0,0 +1,64 @@
|
||||
# 统一导出任务验收记录模板
|
||||
|
||||
## 1. 基本信息
|
||||
|
||||
- 验收日期:
|
||||
- 验收环境:
|
||||
- 验收人:
|
||||
- 代码版本(commit):
|
||||
|
||||
## 2. 请求与响应记录
|
||||
|
||||
### 2.1 创建任务
|
||||
|
||||
- 请求:
|
||||
- 响应:
|
||||
- 期望:
|
||||
- 结果:通过 / 不通过
|
||||
|
||||
### 2.2 列表过滤
|
||||
|
||||
- 请求:
|
||||
- 响应:
|
||||
- 期望:
|
||||
- 结果:通过 / 不通过
|
||||
|
||||
### 2.3 详情下载
|
||||
|
||||
- 请求:
|
||||
- 响应:
|
||||
- 期望:
|
||||
- 结果:通过 / 不通过
|
||||
|
||||
### 2.4 取消任务
|
||||
|
||||
- 请求:
|
||||
- 响应:
|
||||
- 期望:
|
||||
- 结果:通过 / 不通过
|
||||
|
||||
## 3. 数据库快照(PostgreSQL MCP)
|
||||
|
||||
- `tb_export_task` 关键字段快照:
|
||||
- `tb_export_shard_task` 关键字段快照:
|
||||
- 索引核查结果:
|
||||
|
||||
## 4. OSS 产物记录
|
||||
|
||||
- 分片文件 Key 列表:
|
||||
- 最终文件 Key:
|
||||
- 文件大小:
|
||||
|
||||
## 5. 日志关键字段
|
||||
|
||||
- `task_id`:
|
||||
- `task_no`:
|
||||
- `scene`:
|
||||
- `status`:
|
||||
- `error`(如有):
|
||||
|
||||
## 6. 下载 URL 有效期验证
|
||||
|
||||
- 首次获取时间:
|
||||
- 返回 `download_expires_at`:
|
||||
- 过期后刷新结果:
|
||||
84
docs/unified-export-task-system/验证数据样本.md
Normal file
84
docs/unified-export-task-system/验证数据样本.md
Normal file
@@ -0,0 +1,84 @@
|
||||
# 统一导出任务验证数据样本
|
||||
|
||||
## 目标
|
||||
|
||||
为 `device` 与 `iot_card` 场景准备两类样本:
|
||||
|
||||
- 小数据集:便于快速验证状态流转(10~30 条)
|
||||
- 大数据集:便于验证分片执行(>= 5000 条)
|
||||
|
||||
## 1. device 场景
|
||||
|
||||
### 1.1 小数据集(示例)
|
||||
|
||||
```sql
|
||||
-- 建议在测试库执行,按实际字段补齐 creator/updater
|
||||
INSERT INTO tb_device (creator, updater, virtual_no, device_name, device_model, device_type, max_sim_slots, manufacturer, status)
|
||||
VALUES
|
||||
(1, 1, 'EXP-DEV-S-0001', '导出设备样本1', 'M1', 'tracker', 4, 'JH', 1),
|
||||
(1, 1, 'EXP-DEV-S-0002', '导出设备样本2', 'M1', 'tracker', 4, 'JH', 1),
|
||||
(1, 1, 'EXP-DEV-S-0003', '导出设备样本3', 'M2', 'router', 4, 'JH', 1);
|
||||
```
|
||||
|
||||
### 1.2 大数据集(示例)
|
||||
|
||||
```sql
|
||||
-- 生成 5000 条设备样本
|
||||
INSERT INTO tb_device (creator, updater, virtual_no, device_name, device_model, device_type, max_sim_slots, manufacturer, status)
|
||||
SELECT
|
||||
1,
|
||||
1,
|
||||
'EXP-DEV-L-' || LPAD(gs::text, 6, '0'),
|
||||
'导出设备大样本' || gs,
|
||||
'M-L',
|
||||
'tracker',
|
||||
4,
|
||||
'JH',
|
||||
1
|
||||
FROM generate_series(1, 5000) gs;
|
||||
```
|
||||
|
||||
## 2. iot_card 场景
|
||||
|
||||
### 2.1 小数据集(示例)
|
||||
|
||||
```sql
|
||||
INSERT INTO tb_iot_card (
|
||||
creator, updater, iccid, iccid_19, carrier_id, carrier_type, carrier_name,
|
||||
msisdn, status, activation_status, real_name_status, network_status
|
||||
)
|
||||
VALUES
|
||||
(1, 1, '89860000000000000001', '8986000000000000000', 1, 'CMCC', '中国移动', '13900000001', 1, 0, 0, 1),
|
||||
(1, 1, '89860000000000000002', '8986000000000000000', 1, 'CMCC', '中国移动', '13900000002', 1, 0, 0, 1),
|
||||
(1, 1, '89860000000000000003', '8986000000000000000', 1, 'CMCC', '中国移动', '13900000003', 1, 0, 0, 1);
|
||||
```
|
||||
|
||||
### 2.2 大数据集(示例)
|
||||
|
||||
```sql
|
||||
INSERT INTO tb_iot_card (
|
||||
creator, updater, iccid, iccid_19, carrier_id, carrier_type, carrier_name,
|
||||
msisdn, status, activation_status, real_name_status, network_status
|
||||
)
|
||||
SELECT
|
||||
1,
|
||||
1,
|
||||
'8986001' || LPAD(gs::text, 13, '0'),
|
||||
LEFT('8986001' || LPAD(gs::text, 13, '0'), 19),
|
||||
1,
|
||||
'CMCC',
|
||||
'中国移动',
|
||||
'138' || LPAD(gs::text, 8, '0'),
|
||||
1,
|
||||
0,
|
||||
0,
|
||||
1
|
||||
FROM generate_series(1, 5000) gs;
|
||||
```
|
||||
|
||||
## 3. 清理建议
|
||||
|
||||
```sql
|
||||
DELETE FROM tb_device WHERE virtual_no LIKE 'EXP-DEV-%';
|
||||
DELETE FROM tb_iot_card WHERE iccid LIKE '8986001%';
|
||||
```
|
||||
@@ -766,9 +766,12 @@ GET /api/admin/assets/resolve/:identifier
|
||||
|------|------|------|
|
||||
| `real_name_status` | int | 实名状态:0未实名 1实名中 2已实名 |
|
||||
| `current_package` | string | 当前套餐名称(无套餐时为空) |
|
||||
| `package_total_mb` | int64 | 当前套餐总虚流量(MB),已按 `virtual_ratio` 换算 |
|
||||
| `package_used_mb` | float64 | 当前已用虚流量(MB),已按 `virtual_ratio` 换算 |
|
||||
| `package_remain_mb` | float64 | 当前套餐剩余虚流量(MB),已按 `virtual_ratio` 换算 |
|
||||
| `current_package_usage_id` | uint | 当前主套餐的套餐使用记录 ID(无主套餐时为 null) |
|
||||
| `real_total_mb` | int64 | 当前主套餐真实总量(MB) |
|
||||
| `real_used_mb` | int64 | 当前主套餐真实已用量(MB) |
|
||||
| `virtual_total_mb` | int64 | 当前主套餐业务停机阈值(MB) |
|
||||
| `virtual_used_mb` | float64 | 当前主套餐展示已用量(MB) |
|
||||
| `reduction_pct` | float64 | 展示增幅比例,公式为 `(real_total_mb / virtual_total_mb) - 1` |
|
||||
| `device_protect_status` | string | 设备保护期状态:`none`/`stop`/`start`(仅 `asset_type=device`) |
|
||||
|
||||
**绑定关系字段**
|
||||
@@ -825,7 +828,7 @@ GET /api/admin/assets/resolve/:identifier
|
||||
#### Z2 资产实时状态
|
||||
|
||||
```
|
||||
GET /api/admin/assets/:asset_type/:id/realtime-status
|
||||
GET /api/admin/assets/:identifier/realtime-status
|
||||
```
|
||||
|
||||
- **认证**:需要后台 Bearer Token
|
||||
@@ -834,8 +837,7 @@ GET /api/admin/assets/:asset_type/:id/realtime-status
|
||||
|
||||
| 字段 | 类型 | 必填 | 说明 |
|
||||
|------|------|------|------|
|
||||
| `asset_type` | string | 是 | 资产类型:`card` 或 `device` |
|
||||
| `id` | uint | 是 | 资产ID |
|
||||
| `identifier` | string | 是 | 资产标识符(虚拟号/ICCID/IMEI/SN/MSISDN) |
|
||||
|
||||
**响应体(`AssetRealtimeStatusResponse`):**
|
||||
|
||||
@@ -855,7 +857,7 @@ GET /api/admin/assets/:asset_type/:id/realtime-status
|
||||
#### Z3 刷新资产状态
|
||||
|
||||
```
|
||||
POST /api/admin/assets/:asset_type/:id/refresh
|
||||
POST /api/admin/assets/:identifier/refresh
|
||||
```
|
||||
|
||||
- **认证**:需要后台 Bearer Token
|
||||
@@ -868,7 +870,7 @@ POST /api/admin/assets/:asset_type/:id/refresh
|
||||
#### Z4 资产套餐列表
|
||||
|
||||
```
|
||||
GET /api/admin/assets/:asset_type/:id/packages
|
||||
GET /api/admin/assets/:identifier/packages
|
||||
```
|
||||
|
||||
- **认证**:需要后台 Bearer Token
|
||||
@@ -885,12 +887,11 @@ GET /api/admin/assets/:asset_type/:id/packages
|
||||
| `usage_type` | string | 使用类型:`single_card`/`device` |
|
||||
| `status` | int | 状态:0待生效 1生效中 2已用完 3已过期 4已失效 |
|
||||
| `status_name` | string | 状态名称 |
|
||||
| `data_limit_mb` | int64 | 套餐真流量总量(MB) |
|
||||
| `virtual_limit_mb` | int64 | 套餐虚流量总量(MB),按 `virtual_ratio` 换算 |
|
||||
| `data_usage_mb` | int64 | 已用真流量(MB) |
|
||||
| `virtual_used_mb` | float64 | 已用虚流量(MB),按 `virtual_ratio` 换算 |
|
||||
| `virtual_remain_mb` | float64 | 剩余虚流量(MB),按 `virtual_ratio` 换算 |
|
||||
| `virtual_ratio` | float64 | 虚流量比例(real/virtual) |
|
||||
| `real_total_mb` | int64 | 套餐真实总量(MB) |
|
||||
| `real_used_mb` | int64 | 套餐真实已用量(MB) |
|
||||
| `virtual_total_mb` | int64 | 套餐业务停机阈值(MB) |
|
||||
| `virtual_used_mb` | float64 | 套餐展示已用量(MB) |
|
||||
| `reduction_pct` | float64 | 展示增幅比例,公式为 `(real_total_mb / virtual_total_mb) - 1` |
|
||||
| `activated_at` | string | 激活时间(待生效套餐为空) |
|
||||
| `expires_at` | string | 到期时间(待生效套餐为空) |
|
||||
| `master_usage_id` | uint | 主套餐ID(加油包时有值) |
|
||||
@@ -902,12 +903,13 @@ GET /api/admin/assets/:asset_type/:id/packages
|
||||
#### Z5 当前生效套餐
|
||||
|
||||
```
|
||||
GET /api/admin/assets/:asset_type/:id/current-package
|
||||
GET /api/admin/assets/:identifier/current-package
|
||||
```
|
||||
|
||||
- **认证**:需要后台 Bearer Token
|
||||
- **路径参数**:同 Z2
|
||||
- **响应体**:`AssetPackageResponse`(字段同 Z4)
|
||||
- **无主套餐时**:返回 `200 + null`
|
||||
|
||||
---
|
||||
|
||||
|
||||
Reference in New Issue
Block a user