feat: 添加环境变量管理工具和部署配置改版
All checks were successful
构建并部署到测试环境(无 SSH) / build-and-deploy (push) Successful in 5m33s
All checks were successful
构建并部署到测试环境(无 SSH) / build-and-deploy (push) Successful in 5m33s
主要改动: - 新增交互式环境配置脚本 (scripts/setup-env.sh) - 新增本地启动快捷脚本 (scripts/run-local.sh) - 新增环境变量模板文件 (.env.example) - 部署模式改版:使用嵌入式配置 + 环境变量覆盖 - 添加对象存储功能支持 - 改进 IoT 卡片导入任务 - 优化 OpenAPI 文档生成 - 删除旧的配置文件,改用嵌入式默认配置
This commit is contained in:
@@ -64,26 +64,13 @@ jobs:
|
||||
- name: 部署到本地(仅 main 分支)
|
||||
if: github.ref == 'refs/heads/main'
|
||||
run: |
|
||||
# 确保部署目录存在
|
||||
mkdir -p ${{ env.DEPLOY_DIR }}/{configs,logs}
|
||||
|
||||
# 调试:显示当前目录和文件
|
||||
echo "📍 当前工作目录: $(pwd)"
|
||||
echo "📁 当前目录内容:"
|
||||
ls -la
|
||||
# 确保部署目录存在(仅需日志目录,配置已嵌入二进制文件)
|
||||
mkdir -p ${{ env.DEPLOY_DIR }}/logs
|
||||
|
||||
# 强制更新 docker-compose.prod.yml(确保使用最新配置)
|
||||
echo "📋 更新部署配置文件..."
|
||||
cp -v docker-compose.prod.yml ${{ env.DEPLOY_DIR }}/
|
||||
|
||||
# configs 目录只在不存在时初始化(避免覆盖运行时配置)
|
||||
if [ ! -d ${{ env.DEPLOY_DIR }}/configs ] || [ -z "$(ls -A ${{ env.DEPLOY_DIR }}/configs 2>/dev/null)" ]; then
|
||||
echo "📋 初始化配置目录..."
|
||||
cp -rv configs/* ${{ env.DEPLOY_DIR }}/configs/
|
||||
else
|
||||
echo "✅ 配置目录已存在,保留现有配置"
|
||||
fi
|
||||
|
||||
cd ${{ env.DEPLOY_DIR }}
|
||||
|
||||
echo "📥 拉取最新镜像..."
|
||||
|
||||
1
.gitignore
vendored
1
.gitignore
vendored
@@ -75,3 +75,4 @@ __debug_bin1621385388
|
||||
docs/admin-openapi.yaml
|
||||
/api
|
||||
/gendocs
|
||||
.env.local
|
||||
|
||||
44
AGENTS.md
44
AGENTS.md
@@ -151,6 +151,34 @@ Handler → Service → Store → Model
|
||||
- 使用 table-driven tests
|
||||
- 单元测试 < 100ms,集成测试 < 1s
|
||||
|
||||
### ⚠️ 测试真实性原则(严格遵守)
|
||||
|
||||
**测试必须真正验证功能,禁止绕过核心逻辑:**
|
||||
|
||||
| 规则 | 说明 |
|
||||
|------|------|
|
||||
| ❌ 禁止传递 nil 绕过依赖 | 如果功能依赖外部服务(如对象存储、第三方 API),测试必须验证该依赖的调用 |
|
||||
| ❌ 禁止只测试部分流程 | 如果功能包含 A → B → C 三步,不能只测试 B 而跳过 A 和 C |
|
||||
| ❌ 禁止声称"测试通过"但未验证核心逻辑 | 测试通过必须意味着功能真正可用 |
|
||||
| ❌ 禁止擅自使用 Mock | 尽量使用真实服务进行集成测试,如需使用 Mock 必须先询问用户并获得同意 |
|
||||
| ✅ 必须验证端到端流程 | 新增功能必须有完整的集成测试覆盖整个调用链 |
|
||||
| ✅ 缺少配置时必须询问 | 如果测试需要的配置(如 API Key、环境变量)缺失,必须询问用户而非跳过测试 |
|
||||
|
||||
**反面案例**:
|
||||
```go
|
||||
// ❌ 错误:传递 nil 绕过 storageService,只测试了 processImport
|
||||
handler := NewIotCardImportHandler(db, redis, store1, store2, nil, logger)
|
||||
result := handler.processImport(ctx, task) // 跳过了 downloadAndParseCSV
|
||||
|
||||
// ✅ 正确:使用真实服务测试完整流程
|
||||
handler := NewIotCardImportHandler(db, redis, store1, store2, realStorageService, logger)
|
||||
handler.HandleIotCardImport(ctx, asynqTask) // 测试完整流程,验证真实上传/下载
|
||||
```
|
||||
|
||||
**测试超时 = 生产超时**:
|
||||
- 集成测试超时意味着生产环境也可能超时
|
||||
- 发现超时必须排查原因,不能简单跳过或增加超时时间
|
||||
|
||||
### 测试连接管理(必读)
|
||||
|
||||
**详细规范**: [docs/testing/test-connection-guide.md](docs/testing/test-connection-guide.md)
|
||||
@@ -217,4 +245,20 @@ func TestXxx(t *testing.T) {
|
||||
8. ✅ 文档更新计划
|
||||
9. ✅ 中文优先
|
||||
|
||||
### ⚠️ 任务执行规范(必须遵守)
|
||||
|
||||
**提案中的 tasks.md 是契约,不可擅自变更:**
|
||||
|
||||
| 规则 | 说明 |
|
||||
|------|------|
|
||||
| ❌ 禁止跳过任务 | 每个任务都是经过规划的,不能因为"简单"或"显而易见"而跳过 |
|
||||
| ❌ 禁止简化任务 | 不能将多个任务合并或简化执行,除非获得明确许可 |
|
||||
| ❌ 禁止自作主张优化 | 发现可以优化的地方,必须先询问是否可以调整 |
|
||||
| ✅ 必须逐项完成 | 按照 tasks.md 中的顺序逐一执行并标记完成 |
|
||||
| ✅ 必须询问后变更 | 如需调整任务(简化/跳过/合并/优化),先询问用户确认 |
|
||||
|
||||
**询问示例**:
|
||||
> "我注意到任务 2.1 和 2.2 可以合并为一步完成,是否可以这样优化?"
|
||||
> "任务 3.1 在当前实现中可能不需要,是否可以跳过?"
|
||||
|
||||
**详细规范和 OpenSpec 工作流请查看**: `@/openspec/AGENTS.md`
|
||||
|
||||
@@ -59,8 +59,7 @@ WORKDIR /app
|
||||
COPY --from=builder /build/api /app/api
|
||||
COPY --from=builder /go/bin/migrate /usr/local/bin/migrate
|
||||
|
||||
# 复制配置文件和迁移文件
|
||||
COPY configs /app/configs
|
||||
# 复制迁移文件(配置已嵌入二进制文件,无需外部配置文件)
|
||||
COPY migrations /app/migrations
|
||||
|
||||
# 复制启动脚本
|
||||
|
||||
@@ -52,11 +52,11 @@ RUN addgroup -g 1000 appuser && \
|
||||
# 设置工作目录
|
||||
WORKDIR /app
|
||||
|
||||
# 从构建阶段复制二进制文件
|
||||
# 从构建阶段复制二进制文件(配置已嵌入二进制文件,无需外部配置文件)
|
||||
COPY --from=builder /build/worker /app/worker
|
||||
|
||||
# 复制配置文件
|
||||
COPY configs /app/configs
|
||||
# 创建日志目录并设置权限
|
||||
RUN mkdir -p /app/logs && chown -R appuser:appuser /app/logs
|
||||
|
||||
# 切换到非 root 用户
|
||||
USER appuser
|
||||
|
||||
108
README.md
108
README.md
@@ -1,6 +1,6 @@
|
||||
# 君鸿卡管系统 - Fiber 中间件集成
|
||||
|
||||
基于 Go + Fiber 框架的 HTTP 服务,集成了认证、限流、结构化日志和配置热重载功能。
|
||||
基于 Go + Fiber 框架的 HTTP 服务,集成了认证、限流、结构化日志和嵌入式配置功能。
|
||||
|
||||
## 系统简介
|
||||
|
||||
@@ -186,7 +186,7 @@ default:
|
||||
- **认证中间件**:基于 Redis 的 Token 认证
|
||||
- **限流中间件**:基于 IP 的限流,支持可配置的限制和存储后端
|
||||
- **结构化日志**:使用 Zap 的 JSON 日志和自动日志轮转
|
||||
- **配置热重载**:运行时配置更新,无需重启服务
|
||||
- **嵌入式配置**:配置嵌入二进制文件,通过环境变量覆盖,简化 Docker 部署
|
||||
- **请求 ID 追踪**:UUID 跨日志的请求追踪
|
||||
- **Panic 恢复**:优雅的 panic 处理和堆栈跟踪日志
|
||||
- **统一错误处理**:全局 ErrorHandler 统一处理所有 API 错误,返回一致的 JSON 格式(包含错误码、消息、时间戳);Panic 自动恢复防止服务崩溃;错误分类处理(客户端 4xx、服务端 5xx)和日志级别控制;敏感信息自动脱敏保护
|
||||
@@ -199,6 +199,7 @@ default:
|
||||
- **代理商体系**:层级管理和分佣结算
|
||||
- **批量同步**:卡状态、实名状态、流量使用情况
|
||||
- **分佣验证指引**:对代理分佣的冻结、解冻、提现校验流程进行了结构化说明与流程图,详见 [分佣逻辑正确与否验证](docs/优化说明/分佣逻辑正确与否验证.md)
|
||||
- **对象存储**:S3 兼容的对象存储服务集成(联通云 OSS),支持预签名 URL 上传、文件下载、临时文件处理;用于 ICCID 批量导入、数据导出等场景;详见 [使用指南](docs/object-storage/使用指南.md) 和 [前端接入指南](docs/object-storage/前端接入指南.md)
|
||||
|
||||
## 用户体系设计
|
||||
|
||||
@@ -342,13 +343,12 @@ go run cmd/worker/main.go
|
||||
|
||||
**自定义配置**:
|
||||
|
||||
可在 `configs/config.yaml` 中自定义默认管理员信息:
|
||||
通过环境变量自定义默认管理员信息:
|
||||
|
||||
```yaml
|
||||
default_admin:
|
||||
username: "自定义用户名"
|
||||
password: "自定义密码"
|
||||
phone: "自定义手机号"
|
||||
```bash
|
||||
export JUNHONG_DEFAULT_ADMIN_USERNAME="自定义用户名"
|
||||
export JUNHONG_DEFAULT_ADMIN_PASSWORD="自定义密码"
|
||||
export JUNHONG_DEFAULT_ADMIN_PHONE="自定义手机号"
|
||||
```
|
||||
|
||||
**注意事项**:
|
||||
@@ -389,8 +389,9 @@ junhong_cmp_fiber/
|
||||
├── pkg/ # 公共工具库
|
||||
│ ├── config/ # 配置管理
|
||||
│ │ ├── config.go # 配置结构定义
|
||||
│ │ ├── loader.go # 配置加载与验证
|
||||
│ │ └── watcher.go # 配置热重载(fsnotify)
|
||||
│ │ ├── loader.go # 配置加载(嵌入配置 + 环境变量覆盖)
|
||||
│ │ ├── embedded.go # go:embed 嵌入配置加载
|
||||
│ │ └── defaults/config.yaml # 默认配置(嵌入二进制)
|
||||
│ ├── logger/ # 日志基础设施
|
||||
│ │ ├── logger.go # Zap 日志初始化
|
||||
│ │ └── middleware.go # Fiber 日志中间件适配器
|
||||
@@ -408,12 +409,6 @@ junhong_cmp_fiber/
|
||||
│ │ └── redis.go # Redis 客户端初始化
|
||||
│ └── queue/ # 队列封装(Asynq)
|
||||
│
|
||||
├── configs/ # 配置文件
|
||||
│ ├── config.yaml # 默认配置
|
||||
│ ├── config.dev.yaml # 开发环境
|
||||
│ ├── config.staging.yaml # 预发布环境
|
||||
│ └── config.prod.yaml # 生产环境
|
||||
│
|
||||
├── tests/
|
||||
│ └── integration/ # 集成测试
|
||||
│ ├── auth_test.go # 认证测试
|
||||
@@ -630,48 +625,67 @@ KeyAuth:Token 缺失
|
||||
|
||||
## 配置
|
||||
|
||||
### 环境特定配置
|
||||
### 嵌入式配置机制
|
||||
|
||||
设置 `CONFIG_ENV` 环境变量以加载特定配置:
|
||||
系统使用 go:embed 将默认配置嵌入二进制文件,通过环境变量进行覆盖:
|
||||
|
||||
- **默认配置**:`pkg/config/defaults/config.yaml`(编译时嵌入)
|
||||
- **环境变量前缀**:`JUNHONG_`
|
||||
- **格式转换**:配置路径中的 `.` 替换为 `_`
|
||||
|
||||
**环境变量覆盖示例**:
|
||||
|
||||
| 配置项 | 环境变量 |
|
||||
|-------|---------|
|
||||
| `database.host` | `JUNHONG_DATABASE_HOST` |
|
||||
| `redis.address` | `JUNHONG_REDIS_ADDRESS` |
|
||||
| `jwt.secret_key` | `JUNHONG_JWT_SECRET_KEY` |
|
||||
| `logging.level` | `JUNHONG_LOGGING_LEVEL` |
|
||||
|
||||
### 必填配置
|
||||
|
||||
以下配置项必须通过环境变量设置(无默认值或需要覆盖):
|
||||
|
||||
```bash
|
||||
# 开发环境(config.dev.yaml)
|
||||
export CONFIG_ENV=dev
|
||||
# 数据库配置(必填)
|
||||
export JUNHONG_DATABASE_HOST=localhost
|
||||
export JUNHONG_DATABASE_PORT=5432
|
||||
export JUNHONG_DATABASE_USER=postgres
|
||||
export JUNHONG_DATABASE_PASSWORD=your_password
|
||||
export JUNHONG_DATABASE_DBNAME=junhong_cmp
|
||||
|
||||
# 预发布环境(config.staging.yaml)
|
||||
export CONFIG_ENV=staging
|
||||
# Redis 配置(必填)
|
||||
export JUNHONG_REDIS_ADDRESS=localhost
|
||||
|
||||
# 生产环境(config.prod.yaml)
|
||||
export CONFIG_ENV=prod
|
||||
|
||||
# 默认配置(config.yaml)
|
||||
# 不设置 CONFIG_ENV
|
||||
# JWT 密钥(必填,生产环境必须修改)
|
||||
export JUNHONG_JWT_SECRET_KEY=your-secret-key-change-in-production
|
||||
```
|
||||
|
||||
### 配置热重载
|
||||
### Docker 部署
|
||||
|
||||
配置更改在 5 秒内自动检测并应用,无需重启服务器:
|
||||
Docker 部署使用纯环境变量配置,无需挂载配置文件:
|
||||
|
||||
- **监控文件**:所有 `configs/*.yaml` 文件
|
||||
- **检测**:使用 fsnotify 监视文件更改
|
||||
- **验证**:应用前验证新配置
|
||||
- **行为**:
|
||||
- 有效更改:立即应用,记录到 `logs/app.log`
|
||||
- 无效更改:拒绝,服务器继续使用先前配置
|
||||
- **原子性**:使用 `sync/atomic` 进行线程安全的配置更新
|
||||
|
||||
**示例**:
|
||||
```bash
|
||||
# 在服务器运行时编辑配置
|
||||
vim configs/config.yaml
|
||||
# 将 logging.level 从 "info" 改为 "debug"
|
||||
|
||||
# 检查日志(5 秒内)
|
||||
tail -f logs/app.log | jq .
|
||||
# {"level":"info","message":"配置文件已更改","file":"configs/config.yaml"}
|
||||
# {"level":"info","message":"配置重新加载成功"}
|
||||
```yaml
|
||||
# docker-compose.prod.yml 示例
|
||||
services:
|
||||
api:
|
||||
image: registry.boss160.cn/junhong/cmp-fiber-api:latest
|
||||
environment:
|
||||
- JUNHONG_DATABASE_HOST=db-host
|
||||
- JUNHONG_DATABASE_PORT=5432
|
||||
- JUNHONG_DATABASE_USER=postgres
|
||||
- JUNHONG_DATABASE_PASSWORD=secret
|
||||
- JUNHONG_DATABASE_DBNAME=junhong_cmp
|
||||
- JUNHONG_REDIS_ADDRESS=redis
|
||||
- JUNHONG_JWT_SECRET_KEY=production-secret
|
||||
volumes:
|
||||
- ./logs:/app/logs # 仅挂载日志目录
|
||||
```
|
||||
|
||||
### 完整环境变量列表
|
||||
|
||||
详见 [环境变量配置文档](docs/environment-variables.md)
|
||||
|
||||
## 测试
|
||||
|
||||
### 运行所有测试
|
||||
|
||||
@@ -41,6 +41,7 @@ func generateOpenAPIDocs(outputPath string, logger *zap.Logger) {
|
||||
IotCard: admin.NewIotCardHandler(nil),
|
||||
IotCardImport: admin.NewIotCardImportHandler(nil),
|
||||
AssetAllocationRecord: admin.NewAssetAllocationRecordHandler(nil),
|
||||
Storage: admin.NewStorageHandler(nil),
|
||||
}
|
||||
|
||||
// 4. 注册所有路由到文档生成器
|
||||
|
||||
@@ -1,7 +1,6 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"os"
|
||||
"os/signal"
|
||||
"strconv"
|
||||
@@ -22,38 +21,48 @@ import (
|
||||
"github.com/break/junhong_cmp_fiber/internal/routes"
|
||||
"github.com/break/junhong_cmp_fiber/internal/service/verification"
|
||||
"github.com/break/junhong_cmp_fiber/pkg/auth"
|
||||
pkgbootstrap "github.com/break/junhong_cmp_fiber/pkg/bootstrap"
|
||||
"github.com/break/junhong_cmp_fiber/pkg/config"
|
||||
"github.com/break/junhong_cmp_fiber/pkg/database"
|
||||
"github.com/break/junhong_cmp_fiber/pkg/logger"
|
||||
"github.com/break/junhong_cmp_fiber/pkg/queue"
|
||||
"github.com/break/junhong_cmp_fiber/pkg/storage"
|
||||
)
|
||||
|
||||
func main() {
|
||||
// 1. 初始化配置
|
||||
cfg := initConfig()
|
||||
|
||||
// 2. 初始化日志
|
||||
// 2. 初始化目录
|
||||
if _, err := pkgbootstrap.EnsureDirectories(cfg, nil); err != nil {
|
||||
panic("初始化目录失败: " + err.Error())
|
||||
}
|
||||
|
||||
// 3. 初始化日志
|
||||
appLogger := initLogger(cfg)
|
||||
defer func() {
|
||||
_ = logger.Sync()
|
||||
}()
|
||||
|
||||
// 3. 初始化数据库
|
||||
// 4. 初始化数据库
|
||||
db := initDatabase(cfg, appLogger)
|
||||
defer closeDatabase(db, appLogger)
|
||||
|
||||
// 4. 初始化 Redis
|
||||
// 5. 初始化 Redis
|
||||
redisClient := initRedis(cfg, appLogger)
|
||||
defer closeRedis(redisClient, appLogger)
|
||||
|
||||
// 5. 初始化队列客户端
|
||||
// 6. 初始化队列客户端
|
||||
queueClient := initQueue(redisClient, appLogger)
|
||||
defer closeQueue(queueClient, appLogger)
|
||||
|
||||
// 6. 初始化认证管理器
|
||||
// 7. 初始化认证管理器
|
||||
jwtManager, tokenManager, verificationSvc := initAuthComponents(cfg, redisClient, appLogger)
|
||||
|
||||
// 7. 初始化所有业务组件(通过 Bootstrap)
|
||||
// 8. 初始化对象存储服务(可选)
|
||||
storageSvc := initStorage(cfg, appLogger)
|
||||
|
||||
// 9. 初始化所有业务组件(通过 Bootstrap)
|
||||
result, err := bootstrap.Bootstrap(&bootstrap.Dependencies{
|
||||
DB: db,
|
||||
Redis: redisClient,
|
||||
@@ -62,30 +71,26 @@ func main() {
|
||||
TokenManager: tokenManager,
|
||||
VerificationService: verificationSvc,
|
||||
QueueClient: queueClient,
|
||||
StorageService: storageSvc,
|
||||
})
|
||||
if err != nil {
|
||||
appLogger.Fatal("初始化业务组件失败", zap.Error(err))
|
||||
}
|
||||
|
||||
// 8. 启动配置监听器
|
||||
watchCtx, cancelWatch := context.WithCancel(context.Background())
|
||||
defer cancelWatch()
|
||||
go config.Watch(watchCtx, appLogger)
|
||||
|
||||
// 9. 创建 Fiber 应用
|
||||
// 10. 创建 Fiber 应用
|
||||
app := createFiberApp(cfg, appLogger)
|
||||
|
||||
// 10. 注册中间件
|
||||
// 11. 注册中间件
|
||||
initMiddleware(app, cfg, appLogger)
|
||||
|
||||
// 11. 注册路由
|
||||
// 12. 注册路由
|
||||
initRoutes(app, cfg, result, queueClient, db, redisClient, appLogger)
|
||||
|
||||
// 12. 生成 OpenAPI 文档
|
||||
// 13. 生成 OpenAPI 文档
|
||||
generateOpenAPIDocs("logs/openapi.yaml", appLogger)
|
||||
|
||||
// 13. 启动服务器
|
||||
startServer(app, cfg, appLogger, cancelWatch)
|
||||
// 14. 启动服务器
|
||||
startServer(app, cfg, appLogger)
|
||||
}
|
||||
|
||||
// initConfig 加载配置
|
||||
@@ -257,9 +262,7 @@ func initRateLimiter(router fiber.Router, cfg *config.Config, appLogger *zap.Log
|
||||
))
|
||||
}
|
||||
|
||||
// startServer 启动服务器
|
||||
func startServer(app *fiber.App, cfg *config.Config, appLogger *zap.Logger, cancelWatch context.CancelFunc) {
|
||||
// 优雅关闭
|
||||
func startServer(app *fiber.App, cfg *config.Config, appLogger *zap.Logger) {
|
||||
quit := make(chan os.Signal, 1)
|
||||
signal.Notify(quit, os.Interrupt, syscall.SIGTERM)
|
||||
|
||||
@@ -271,14 +274,9 @@ func startServer(app *fiber.App, cfg *config.Config, appLogger *zap.Logger, canc
|
||||
|
||||
appLogger.Info("服务器已启动", zap.String("address", cfg.Server.Address))
|
||||
|
||||
// 等待关闭信号
|
||||
<-quit
|
||||
appLogger.Info("正在关闭服务器...")
|
||||
|
||||
// 取消配置监听器
|
||||
cancelWatch()
|
||||
|
||||
// 关闭 HTTP 服务器
|
||||
if err := app.ShutdownWithTimeout(cfg.Server.ShutdownTimeout); err != nil {
|
||||
appLogger.Error("强制关闭服务器", zap.Error(err))
|
||||
}
|
||||
@@ -297,3 +295,23 @@ func initAuthComponents(cfg *config.Config, redisClient *redis.Client, appLogger
|
||||
|
||||
return jwtManager, tokenManager, verificationSvc
|
||||
}
|
||||
|
||||
func initStorage(cfg *config.Config, appLogger *zap.Logger) *storage.Service {
|
||||
if cfg.Storage.Provider == "" || cfg.Storage.S3.Endpoint == "" {
|
||||
appLogger.Info("对象存储未配置,跳过初始化")
|
||||
return nil
|
||||
}
|
||||
|
||||
provider, err := storage.NewS3Provider(&cfg.Storage)
|
||||
if err != nil {
|
||||
appLogger.Warn("初始化对象存储失败,功能将不可用", zap.Error(err))
|
||||
return nil
|
||||
}
|
||||
|
||||
appLogger.Info("对象存储已初始化",
|
||||
zap.String("provider", cfg.Storage.Provider),
|
||||
zap.String("bucket", cfg.Storage.S3.Bucket),
|
||||
)
|
||||
|
||||
return storage.NewService(provider, &cfg.Storage)
|
||||
}
|
||||
|
||||
@@ -50,6 +50,7 @@ func generateAdminDocs(outputPath string) error {
|
||||
IotCard: admin.NewIotCardHandler(nil),
|
||||
IotCardImport: admin.NewIotCardImportHandler(nil),
|
||||
AssetAllocationRecord: admin.NewAssetAllocationRecordHandler(nil),
|
||||
Storage: admin.NewStorageHandler(nil),
|
||||
}
|
||||
|
||||
// 4. 注册所有路由到文档生成器
|
||||
|
||||
@@ -10,20 +10,24 @@ import (
|
||||
"github.com/redis/go-redis/v9"
|
||||
"go.uber.org/zap"
|
||||
|
||||
"github.com/break/junhong_cmp_fiber/pkg/bootstrap"
|
||||
"github.com/break/junhong_cmp_fiber/pkg/config"
|
||||
"github.com/break/junhong_cmp_fiber/pkg/database"
|
||||
"github.com/break/junhong_cmp_fiber/pkg/logger"
|
||||
"github.com/break/junhong_cmp_fiber/pkg/queue"
|
||||
"github.com/break/junhong_cmp_fiber/pkg/storage"
|
||||
)
|
||||
|
||||
func main() {
|
||||
// 加载配置
|
||||
cfg, err := config.Load()
|
||||
if err != nil {
|
||||
panic("加载配置失败: " + err.Error())
|
||||
}
|
||||
|
||||
// 初始化日志
|
||||
if _, err := bootstrap.EnsureDirectories(cfg, nil); err != nil {
|
||||
panic("初始化目录失败: " + err.Error())
|
||||
}
|
||||
|
||||
if err := logger.InitLoggers(
|
||||
cfg.Logging.Level,
|
||||
cfg.Logging.Development,
|
||||
@@ -90,11 +94,14 @@ func main() {
|
||||
}
|
||||
}()
|
||||
|
||||
// 初始化对象存储服务(可选)
|
||||
storageSvc := initStorage(cfg, appLogger)
|
||||
|
||||
// 创建 Asynq Worker 服务器
|
||||
workerServer := queue.NewServer(redisClient, &cfg.Queue, appLogger)
|
||||
|
||||
// 创建任务处理器管理器并注册所有处理器
|
||||
taskHandler := queue.NewHandler(db, redisClient, appLogger)
|
||||
taskHandler := queue.NewHandler(db, redisClient, storageSvc, appLogger)
|
||||
taskHandler.RegisterHandlers()
|
||||
|
||||
appLogger.Info("Worker 服务器配置完成",
|
||||
@@ -123,3 +130,23 @@ func main() {
|
||||
|
||||
appLogger.Info("Worker 服务器已停止")
|
||||
}
|
||||
|
||||
func initStorage(cfg *config.Config, appLogger *zap.Logger) *storage.Service {
|
||||
if cfg.Storage.Provider == "" || cfg.Storage.S3.Endpoint == "" {
|
||||
appLogger.Info("对象存储未配置,跳过初始化")
|
||||
return nil
|
||||
}
|
||||
|
||||
provider, err := storage.NewS3Provider(&cfg.Storage)
|
||||
if err != nil {
|
||||
appLogger.Warn("初始化对象存储失败,功能将不可用", zap.Error(err))
|
||||
return nil
|
||||
}
|
||||
|
||||
appLogger.Info("对象存储已初始化",
|
||||
zap.String("provider", cfg.Storage.Provider),
|
||||
zap.String("bucket", cfg.Storage.S3.Bucket),
|
||||
)
|
||||
|
||||
return storage.NewService(provider, &cfg.Storage)
|
||||
}
|
||||
|
||||
@@ -1,84 +0,0 @@
|
||||
server:
|
||||
address: ":3000"
|
||||
read_timeout: "10s"
|
||||
write_timeout: "10s"
|
||||
shutdown_timeout: "30s"
|
||||
prefork: false
|
||||
|
||||
redis:
|
||||
address: "cxd.whcxd.cn"
|
||||
password: "cpNbWtAaqgo1YJmbMp3h"
|
||||
port: 16299
|
||||
db: 0
|
||||
pool_size: 10
|
||||
min_idle_conns: 5
|
||||
dial_timeout: "5s"
|
||||
read_timeout: "3s"
|
||||
write_timeout: "3s"
|
||||
|
||||
database:
|
||||
host: "cxd.whcxd.cn"
|
||||
port: 16159
|
||||
user: "erp_pgsql"
|
||||
password: "erp_2025"
|
||||
dbname: "junhong_cmp_test"
|
||||
sslmode: "disable"
|
||||
max_open_conns: 25
|
||||
max_idle_conns: 10
|
||||
conn_max_lifetime: "5m"
|
||||
|
||||
queue:
|
||||
concurrency: 10
|
||||
queues:
|
||||
critical: 6
|
||||
default: 3
|
||||
low: 1
|
||||
retry_max: 5
|
||||
timeout: "10m"
|
||||
|
||||
logging:
|
||||
level: "debug" # 开发环境使用 debug 级别
|
||||
development: true # 启用开发模式(美化日志输出)
|
||||
app_log:
|
||||
filename: "logs/app.log"
|
||||
max_size: 100
|
||||
max_backups: 10 # 开发环境保留较少备份
|
||||
max_age: 7 # 7天
|
||||
compress: false # 开发环境不压缩
|
||||
access_log:
|
||||
filename: "logs/access.log"
|
||||
max_size: 100
|
||||
max_backups: 10
|
||||
max_age: 7
|
||||
compress: false
|
||||
|
||||
middleware:
|
||||
enable_rate_limiter: true
|
||||
rate_limiter:
|
||||
max: 1000
|
||||
expiration: "1m"
|
||||
storage: "redis"
|
||||
|
||||
sms:
|
||||
gateway_url: "https://gateway.sms.whjhft.com:8443/sms"
|
||||
username: "JH0001" # TODO: 替换为实际的短信服务账号
|
||||
password: "wwR8E4qnL6F0" # TODO: 替换为实际的短信服务密码
|
||||
signature: "【JHFTIOT】" # TODO: 替换为报备通过的短信签名
|
||||
timeout: "10s"
|
||||
|
||||
# JWT 配置
|
||||
jwt:
|
||||
secret_key: "dev-secret-key-for-testing-only-32chars!"
|
||||
token_duration: "168h" # C 端个人客户 JWT Token 有效期(7天)
|
||||
access_token_ttl: "24h" # B 端访问令牌有效期(24小时)
|
||||
refresh_token_ttl: "168h" # B 端刷新令牌有效期(7天)
|
||||
|
||||
# 默认超级管理员配置(可选,系统启动时自动创建)
|
||||
# 如果配置为空,系统使用代码默认值:
|
||||
# - 用户名: admin
|
||||
# - 密码: Admin@123456
|
||||
# - 手机号: 13800000000
|
||||
# default_admin:
|
||||
# username: "admin"
|
||||
# password: "Admin@123456"
|
||||
# phone: "13800000000"
|
||||
@@ -1,72 +0,0 @@
|
||||
server:
|
||||
address: ":8080"
|
||||
read_timeout: "10s"
|
||||
write_timeout: "10s"
|
||||
shutdown_timeout: "30s"
|
||||
prefork: true # 生产环境启用多进程模式
|
||||
|
||||
redis:
|
||||
address: "redis-prod:6379"
|
||||
password: "${REDIS_PASSWORD}"
|
||||
db: 0
|
||||
pool_size: 50 # 生产环境更大的连接池
|
||||
min_idle_conns: 20
|
||||
dial_timeout: "5s"
|
||||
read_timeout: "3s"
|
||||
write_timeout: "3s"
|
||||
|
||||
database:
|
||||
host: "postgres-prod"
|
||||
port: 5432
|
||||
user: "postgres"
|
||||
password: "${DB_PASSWORD}" # 从环境变量读取
|
||||
dbname: "junhong_cmp"
|
||||
sslmode: "require" # 生产环境必须启用 SSL
|
||||
max_open_conns: 50 # 生产环境更大的连接池
|
||||
max_idle_conns: 20
|
||||
conn_max_lifetime: "5m"
|
||||
|
||||
queue:
|
||||
concurrency: 20 # 生产环境更高并发
|
||||
queues:
|
||||
critical: 6
|
||||
default: 3
|
||||
low: 1
|
||||
retry_max: 5
|
||||
timeout: "10m"
|
||||
|
||||
logging:
|
||||
level: "warn" # 生产环境较少详细日志
|
||||
development: false
|
||||
app_log:
|
||||
filename: "logs/app.log"
|
||||
max_size: 100
|
||||
max_backups: 60
|
||||
max_age: 60
|
||||
compress: true
|
||||
access_log:
|
||||
filename: "logs/access.log"
|
||||
max_size: 500
|
||||
max_backups: 180
|
||||
max_age: 180
|
||||
compress: true
|
||||
|
||||
middleware:
|
||||
# 生产环境启用限流,保护服务免受滥用
|
||||
enable_rate_limiter: true
|
||||
|
||||
# 限流器配置(生产环境)
|
||||
rate_limiter:
|
||||
# 生产环境限制:每分钟5000请求
|
||||
# 根据实际业务需求调整
|
||||
max: 5000
|
||||
|
||||
# 1分钟窗口(标准配置)
|
||||
expiration: "1m"
|
||||
|
||||
# 生产环境使用 Redis 分布式限流
|
||||
# 优势:
|
||||
# 1. 多服务器实例共享限流计数器
|
||||
# 2. 限流状态持久化,服务重启不丢失
|
||||
# 3. 精确的全局限流控制
|
||||
storage: "redis"
|
||||
@@ -1,68 +0,0 @@
|
||||
server:
|
||||
address: ":8080"
|
||||
read_timeout: "10s"
|
||||
write_timeout: "10s"
|
||||
shutdown_timeout: "30s"
|
||||
prefork: false
|
||||
|
||||
redis:
|
||||
address: "redis-staging:6379"
|
||||
password: "${REDIS_PASSWORD}" # 从环境变量读取
|
||||
db: 0
|
||||
pool_size: 20
|
||||
min_idle_conns: 10
|
||||
dial_timeout: "5s"
|
||||
read_timeout: "3s"
|
||||
write_timeout: "3s"
|
||||
|
||||
database:
|
||||
host: "postgres-staging"
|
||||
port: 5432
|
||||
user: "postgres"
|
||||
password: "${DB_PASSWORD}" # 从环境变量读取
|
||||
dbname: "junhong_cmp_staging"
|
||||
sslmode: "require" # 预发布环境启用 SSL
|
||||
max_open_conns: 25
|
||||
max_idle_conns: 10
|
||||
conn_max_lifetime: "5m"
|
||||
|
||||
queue:
|
||||
concurrency: 10
|
||||
queues:
|
||||
critical: 6
|
||||
default: 3
|
||||
low: 1
|
||||
retry_max: 5
|
||||
timeout: "10m"
|
||||
|
||||
logging:
|
||||
level: "info"
|
||||
development: false
|
||||
app_log:
|
||||
filename: "logs/app.log"
|
||||
max_size: 100
|
||||
max_backups: 30
|
||||
max_age: 30
|
||||
compress: true
|
||||
access_log:
|
||||
filename: "logs/access.log"
|
||||
max_size: 500
|
||||
max_backups: 90
|
||||
max_age: 90
|
||||
compress: true
|
||||
|
||||
middleware:
|
||||
# 预发布环境启用限流,测试生产配置
|
||||
enable_rate_limiter: true
|
||||
|
||||
# 限流器配置(预发布环境)
|
||||
rate_limiter:
|
||||
# 预发布环境使用中等限制,模拟生产负载
|
||||
max: 1000
|
||||
|
||||
# 1分钟窗口
|
||||
expiration: "1m"
|
||||
|
||||
# 预发布环境可使用内存存储(简化测试)
|
||||
# 如果需要测试分布式限流,改为 "redis"
|
||||
storage: "memory"
|
||||
@@ -1,84 +0,0 @@
|
||||
server:
|
||||
address: ":3000"
|
||||
read_timeout: "10s"
|
||||
write_timeout: "10s"
|
||||
shutdown_timeout: "30s"
|
||||
prefork: false
|
||||
|
||||
redis:
|
||||
address: "cxd.whcxd.cn"
|
||||
password: "cpNbWtAaqgo1YJmbMp3h"
|
||||
port: 16299
|
||||
db: 6
|
||||
pool_size: 10
|
||||
min_idle_conns: 5
|
||||
dial_timeout: "5s"
|
||||
read_timeout: "3s"
|
||||
write_timeout: "3s"
|
||||
|
||||
database:
|
||||
host: "cxd.whcxd.cn"
|
||||
port: 16159
|
||||
user: "erp_pgsql"
|
||||
password: "erp_2025"
|
||||
dbname: "junhong_cmp_test"
|
||||
sslmode: "disable"
|
||||
max_open_conns: 25
|
||||
max_idle_conns: 10
|
||||
conn_max_lifetime: "5m"
|
||||
|
||||
queue:
|
||||
concurrency: 10
|
||||
queues:
|
||||
critical: 6
|
||||
default: 3
|
||||
low: 1
|
||||
retry_max: 5
|
||||
timeout: "10m"
|
||||
|
||||
logging:
|
||||
level: "debug" # 开发环境使用 debug 级别
|
||||
development: true # 启用开发模式(美化日志输出)
|
||||
app_log:
|
||||
filename: "logs/app.log"
|
||||
max_size: 100
|
||||
max_backups: 10 # 开发环境保留较少备份
|
||||
max_age: 7 # 7天
|
||||
compress: false # 开发环境不压缩
|
||||
access_log:
|
||||
filename: "logs/access.log"
|
||||
max_size: 100
|
||||
max_backups: 10
|
||||
max_age: 7
|
||||
compress: false
|
||||
|
||||
middleware:
|
||||
enable_rate_limiter: true
|
||||
rate_limiter:
|
||||
max: 1000
|
||||
expiration: "1m"
|
||||
storage: "redis"
|
||||
|
||||
sms:
|
||||
gateway_url: "https://gateway.sms.whjhft.com:8443/sms"
|
||||
username: "JH0001" # TODO: 替换为实际的短信服务账号
|
||||
password: "wwR8E4qnL6F0" # TODO: 替换为实际的短信服务密码
|
||||
signature: "【JHFTIOT】" # TODO: 替换为报备通过的短信签名
|
||||
timeout: "10s"
|
||||
|
||||
# JWT 配置
|
||||
jwt:
|
||||
secret_key: "dev-secret-key-for-testing-only-32chars!"
|
||||
token_duration: "168h" # C 端个人客户 JWT Token 有效期(7天)
|
||||
access_token_ttl: "24h" # B 端访问令牌有效期(24小时)
|
||||
refresh_token_ttl: "168h" # B 端刷新令牌有效期(7天)
|
||||
|
||||
# 默认超级管理员配置(可选,系统启动时自动创建)
|
||||
# 如果配置为空,系统使用代码默认值:
|
||||
# - 用户名: admin
|
||||
# - 密码: Admin@123456
|
||||
# - 手机号: 13800000000
|
||||
# default_admin:
|
||||
# username: "admin"
|
||||
# password: "Admin@123456"
|
||||
# phone: "13800000000"
|
||||
@@ -1,5 +1,25 @@
|
||||
version: '3.8'
|
||||
|
||||
# 君鸿卡管系统生产环境部署配置
|
||||
#
|
||||
# 配置方式:纯环境变量配置(配置已嵌入二进制文件)
|
||||
# 环境变量前缀:JUNHONG_
|
||||
# 格式:JUNHONG_{配置路径},路径分隔符用下划线替代点号
|
||||
#
|
||||
# 示例:
|
||||
# database.host → JUNHONG_DATABASE_HOST
|
||||
# redis.address → JUNHONG_REDIS_ADDRESS
|
||||
# jwt.secret_key → JUNHONG_JWT_SECRET_KEY
|
||||
#
|
||||
# 必填配置(缺失时服务无法启动):
|
||||
# - JUNHONG_DATABASE_HOST
|
||||
# - JUNHONG_DATABASE_PORT
|
||||
# - JUNHONG_DATABASE_USER
|
||||
# - JUNHONG_DATABASE_PASSWORD
|
||||
# - JUNHONG_DATABASE_DBNAME
|
||||
# - JUNHONG_REDIS_ADDRESS
|
||||
# - JUNHONG_JWT_SECRET_KEY
|
||||
|
||||
services:
|
||||
api:
|
||||
image: registry.boss160.cn/junhong/cmp-fiber-api:latest
|
||||
@@ -8,19 +28,39 @@ services:
|
||||
ports:
|
||||
- "3000:3000"
|
||||
environment:
|
||||
- DB_HOST=cxd.whcxd.cn
|
||||
- DB_PORT=16159
|
||||
- DB_USER=erp_pgsql
|
||||
- DB_PASSWORD=erp_2025
|
||||
- DB_NAME=junhong_cmp_test
|
||||
- DB_SSLMODE=disable
|
||||
# 数据库配置(必填)
|
||||
- JUNHONG_DATABASE_HOST=cxd.whcxd.cn
|
||||
- JUNHONG_DATABASE_PORT=16159
|
||||
- JUNHONG_DATABASE_USER=erp_pgsql
|
||||
- JUNHONG_DATABASE_PASSWORD=erp_2025
|
||||
- JUNHONG_DATABASE_DBNAME=junhong_cmp_test
|
||||
- JUNHONG_DATABASE_SSLMODE=disable
|
||||
# Redis 配置(必填)
|
||||
- JUNHONG_REDIS_ADDRESS=cxd.whcxd.cn
|
||||
- JUNHONG_REDIS_PORT=16299
|
||||
- JUNHONG_REDIS_PASSWORD=cpNbWtAaqgo1YJmbMp3h
|
||||
- JUNHONG_REDIS_DB=6
|
||||
# JWT 配置(必填)
|
||||
- JUNHONG_JWT_SECRET_KEY=dev-secret-key-for-testing-only-32chars!
|
||||
# 日志配置
|
||||
- JUNHONG_LOGGING_LEVEL=info
|
||||
- JUNHONG_LOGGING_DEVELOPMENT=false
|
||||
# 对象存储配置
|
||||
- JUNHONG_STORAGE_PROVIDER=s3
|
||||
- JUNHONG_STORAGE_S3_ENDPOINT=http://obs-helf.cucloud.cn
|
||||
- JUNHONG_STORAGE_S3_REGION=cn-langfang-2
|
||||
- JUNHONG_STORAGE_S3_BUCKET=cmp
|
||||
- JUNHONG_STORAGE_S3_ACCESS_KEY_ID=598F558CF6FF46E79D1CFC607852378C9523
|
||||
- JUNHONG_STORAGE_S3_SECRET_ACCESS_KEY=8393425DCB2F48F1914FF39DCBC6C7B17325
|
||||
- JUNHONG_STORAGE_S3_USE_SSL=false
|
||||
- JUNHONG_STORAGE_S3_PATH_STYLE=true
|
||||
volumes:
|
||||
- ./configs:/app/configs:ro
|
||||
# 仅挂载日志目录(配置已嵌入二进制文件)
|
||||
- ./logs:/app/logs
|
||||
networks:
|
||||
- junhong-network
|
||||
healthcheck:
|
||||
test: ["CMD", "wget", "--no-verbose", "--tries=1", "--spider", "http://127.0.0.1:3000/health"]
|
||||
test: [ "CMD", "wget", "--no-verbose", "--tries=1", "--spider", "http://127.0.0.1:3000/health" ]
|
||||
interval: 30s
|
||||
timeout: 3s
|
||||
retries: 3
|
||||
@@ -35,8 +75,34 @@ services:
|
||||
image: registry.boss160.cn/junhong/cmp-fiber-worker:latest
|
||||
container_name: junhong-cmp-worker
|
||||
restart: unless-stopped
|
||||
environment:
|
||||
# 数据库配置(必填)
|
||||
- JUNHONG_DATABASE_HOST=cxd.whcxd.cn
|
||||
- JUNHONG_DATABASE_PORT=16159
|
||||
- JUNHONG_DATABASE_USER=erp_pgsql
|
||||
- JUNHONG_DATABASE_PASSWORD=erp_2025
|
||||
- JUNHONG_DATABASE_DBNAME=junhong_cmp_test
|
||||
- JUNHONG_DATABASE_SSLMODE=disable
|
||||
# Redis 配置(必填)
|
||||
- JUNHONG_REDIS_ADDRESS=cxd.whcxd.cn
|
||||
- JUNHONG_REDIS_PORT=16299
|
||||
- JUNHONG_REDIS_PASSWORD=cpNbWtAaqgo1YJmbMp3h
|
||||
- JUNHONG_REDIS_DB=6
|
||||
# JWT 配置(必填)
|
||||
- JUNHONG_JWT_SECRET_KEY=dev-secret-key-for-testing-only-32chars!
|
||||
# 日志配置
|
||||
- JUNHONG_LOGGING_LEVEL=info
|
||||
- JUNHONG_LOGGING_DEVELOPMENT=false
|
||||
# 对象存储配置
|
||||
- JUNHONG_STORAGE_PROVIDER=s3
|
||||
- JUNHONG_STORAGE_S3_ENDPOINT=http://obs-helf.cucloud.cn
|
||||
- JUNHONG_STORAGE_S3_REGION=cn-langfang-2
|
||||
- JUNHONG_STORAGE_S3_BUCKET=cmp
|
||||
- JUNHONG_STORAGE_S3_ACCESS_KEY_ID=598F558CF6FF46E79D1CFC607852378C9523
|
||||
- JUNHONG_STORAGE_S3_SECRET_ACCESS_KEY=8393425DCB2F48F1914FF39DCBC6C7B17325
|
||||
- JUNHONG_STORAGE_S3_USE_SSL=false
|
||||
- JUNHONG_STORAGE_S3_PATH_STYLE=true
|
||||
volumes:
|
||||
- ./configs:/app/configs:ro
|
||||
- ./logs:/app/logs
|
||||
networks:
|
||||
- junhong-network
|
||||
|
||||
@@ -5,17 +5,18 @@ echo "========================================="
|
||||
echo "君鸿卡管系统 API 服务启动中..."
|
||||
echo "========================================="
|
||||
|
||||
# 检查必要的环境变量
|
||||
if [ -z "$DB_HOST" ]; then
|
||||
echo "错误: DB_HOST 环境变量未设置"
|
||||
exit 1
|
||||
fi
|
||||
# 构建数据库连接 URL(从环境变量读取)
|
||||
# 环境变量由 docker-compose 传入,格式为 JUNHONG_DATABASE_*
|
||||
DB_HOST="${JUNHONG_DATABASE_HOST:-localhost}"
|
||||
DB_PORT="${JUNHONG_DATABASE_PORT:-5432}"
|
||||
DB_USER="${JUNHONG_DATABASE_USER:-postgres}"
|
||||
DB_PASSWORD="${JUNHONG_DATABASE_PASSWORD:-}"
|
||||
DB_NAME="${JUNHONG_DATABASE_DBNAME:-junhong_cmp}"
|
||||
DB_SSLMODE="${JUNHONG_DATABASE_SSLMODE:-disable}"
|
||||
|
||||
# 构建数据库连接 URL
|
||||
DB_URL="postgresql://${DB_USER}:${DB_PASSWORD}@${DB_HOST}:${DB_PORT}/${DB_NAME}?sslmode=${DB_SSLMODE}"
|
||||
|
||||
echo "检查数据库连接..."
|
||||
# 等待数据库就绪(最多等待 30 秒)
|
||||
for i in {1..30}; do
|
||||
if migrate -path /app/migrations -database "$DB_URL" version > /dev/null 2>&1; then
|
||||
echo "数据库连接成功"
|
||||
@@ -25,7 +26,6 @@ for i in {1..30}; do
|
||||
sleep 1
|
||||
done
|
||||
|
||||
# 执行数据库迁移
|
||||
echo "执行数据库迁移..."
|
||||
if migrate -path /app/migrations -database "$DB_URL" up; then
|
||||
echo "数据库迁移完成"
|
||||
@@ -33,7 +33,6 @@ else
|
||||
echo "警告: 数据库迁移失败或无新迁移"
|
||||
fi
|
||||
|
||||
# 启动 API 服务
|
||||
echo "启动 API 服务..."
|
||||
echo "========================================="
|
||||
exec /app/api
|
||||
|
||||
@@ -140,17 +140,16 @@ if err := initDefaultAdmin(deps, services); err != nil {
|
||||
|
||||
### 自定义配置
|
||||
|
||||
在 `configs/config.yaml` 中添加:
|
||||
通过环境变量自定义:
|
||||
|
||||
```yaml
|
||||
default_admin:
|
||||
username: "自定义用户名"
|
||||
password: "自定义密码"
|
||||
phone: "自定义手机号"
|
||||
```bash
|
||||
export JUNHONG_DEFAULT_ADMIN_USERNAME="自定义用户名"
|
||||
export JUNHONG_DEFAULT_ADMIN_PASSWORD="自定义密码"
|
||||
export JUNHONG_DEFAULT_ADMIN_PHONE="自定义手机号"
|
||||
```
|
||||
|
||||
**注意**:
|
||||
- 配置项为可选,不参与 `Validate()` 验证
|
||||
- 配置项为可选,不参与 `ValidateRequired()` 验证
|
||||
- 任何字段留空则使用代码默认值
|
||||
- 密码必须足够复杂(建议包含大小写字母、数字、特殊字符)
|
||||
|
||||
@@ -192,12 +191,11 @@ go run cmd/api/main.go
|
||||
|
||||
### 场景3:使用自定义配置
|
||||
|
||||
**配置文件** (`configs/config.yaml`):
|
||||
```yaml
|
||||
default_admin:
|
||||
username: "myadmin"
|
||||
password: "MySecurePass@2024"
|
||||
phone: "13900000000"
|
||||
**设置环境变量**:
|
||||
```bash
|
||||
export JUNHONG_DEFAULT_ADMIN_USERNAME="myadmin"
|
||||
export JUNHONG_DEFAULT_ADMIN_PASSWORD="MySecurePass@2024"
|
||||
export JUNHONG_DEFAULT_ADMIN_PHONE="13900000000"
|
||||
```
|
||||
|
||||
**启动服务**:
|
||||
@@ -230,11 +228,11 @@ go run cmd/api/main.go
|
||||
- ✅ 包含时间戳、用户名、手机号
|
||||
- ⚠️ 日志中不会记录明文密码
|
||||
|
||||
### 4. 配置文件安全
|
||||
### 4. 配置安全
|
||||
|
||||
- ⚠️ `config.yaml` 中的密码是明文存储
|
||||
- ⚠️ 确保配置文件访问权限受限(不要提交到公开仓库)
|
||||
- ⚠️ 生产环境建议使用环境变量或密钥管理服务
|
||||
- ✅ 配置通过环境变量设置,不存储在代码仓库中
|
||||
- ⚠️ 确保环境变量安全(使用密钥管理服务或加密存储)
|
||||
- ⚠️ 生产环境务必修改默认密码
|
||||
|
||||
## 手动创建管理员(备用方案)
|
||||
|
||||
@@ -285,7 +283,8 @@ func main() {
|
||||
|
||||
- `pkg/constants/constants.go` - 默认值常量定义
|
||||
- `pkg/config/config.go` - 配置结构定义
|
||||
- `configs/config.yaml` - 配置示例
|
||||
- `pkg/config/defaults/config.yaml` - 嵌入式默认配置
|
||||
- `docs/environment-variables.md` - 环境变量配置文档
|
||||
- `internal/service/account/service.go` - CreateSystemAccount 方法
|
||||
- `internal/bootstrap/admin.go` - initDefaultAdmin 函数
|
||||
- `internal/bootstrap/bootstrap.go` - Bootstrap 主流程
|
||||
|
||||
@@ -148,6 +148,77 @@ components:
|
||||
description: 总卡数量
|
||||
type: integer
|
||||
type: object
|
||||
DtoAllocateStandaloneCardsRequest:
|
||||
properties:
|
||||
batch_no:
|
||||
description: 批次号(selection_type=filter时可选)
|
||||
maxLength: 100
|
||||
type: string
|
||||
carrier_id:
|
||||
description: 运营商ID(selection_type=filter时可选)
|
||||
minimum: 0
|
||||
nullable: true
|
||||
type: integer
|
||||
iccid_end:
|
||||
description: 结束ICCID(selection_type=range时必填)
|
||||
maxLength: 20
|
||||
type: string
|
||||
iccid_start:
|
||||
description: 起始ICCID(selection_type=range时必填)
|
||||
maxLength: 20
|
||||
type: string
|
||||
iccids:
|
||||
description: ICCID列表(selection_type=list时必填,最多1000个)
|
||||
items:
|
||||
type: string
|
||||
nullable: true
|
||||
type: array
|
||||
remark:
|
||||
description: 备注
|
||||
maxLength: 500
|
||||
type: string
|
||||
selection_type:
|
||||
description: 选卡方式 (list:ICCID列表, range:号段范围, filter:筛选条件)
|
||||
enum:
|
||||
- list
|
||||
- range
|
||||
- filter
|
||||
type: string
|
||||
status:
|
||||
description: 卡状态 (1:在库, 2:已分销)(selection_type=filter时可选)
|
||||
maximum: 4
|
||||
minimum: 1
|
||||
nullable: true
|
||||
type: integer
|
||||
to_shop_id:
|
||||
description: 目标店铺ID
|
||||
minimum: 1
|
||||
type: integer
|
||||
required:
|
||||
- to_shop_id
|
||||
- selection_type
|
||||
type: object
|
||||
DtoAllocateStandaloneCardsResponse:
|
||||
properties:
|
||||
allocation_no:
|
||||
description: 分配单号
|
||||
type: string
|
||||
fail_count:
|
||||
description: 失败数
|
||||
type: integer
|
||||
failed_items:
|
||||
description: 失败项列表
|
||||
items:
|
||||
$ref: '#/components/schemas/DtoAllocationFailedItem'
|
||||
nullable: true
|
||||
type: array
|
||||
success_count:
|
||||
description: 成功数
|
||||
type: integer
|
||||
total_count:
|
||||
description: 待分配总数
|
||||
type: integer
|
||||
type: object
|
||||
DtoAllocatedDevice:
|
||||
properties:
|
||||
card_count:
|
||||
@@ -167,6 +238,15 @@ components:
|
||||
nullable: true
|
||||
type: array
|
||||
type: object
|
||||
DtoAllocationFailedItem:
|
||||
properties:
|
||||
iccid:
|
||||
description: ICCID
|
||||
type: string
|
||||
reason:
|
||||
description: 失败原因
|
||||
type: string
|
||||
type: object
|
||||
DtoApproveWithdrawalReq:
|
||||
properties:
|
||||
account_name:
|
||||
@@ -198,6 +278,156 @@ components:
|
||||
required:
|
||||
- payment_type
|
||||
type: object
|
||||
DtoAssetAllocationRecordDetailResponse:
|
||||
properties:
|
||||
allocation_name:
|
||||
description: 分配类型名称
|
||||
type: string
|
||||
allocation_no:
|
||||
description: 分配单号
|
||||
type: string
|
||||
allocation_type:
|
||||
description: 分配类型 (allocate:分配, recall:回收)
|
||||
type: string
|
||||
asset_id:
|
||||
description: 资产ID
|
||||
minimum: 0
|
||||
type: integer
|
||||
asset_identifier:
|
||||
description: 资产标识符(ICCID或设备号)
|
||||
type: string
|
||||
asset_type:
|
||||
description: 资产类型 (iot_card:物联网卡, device:设备)
|
||||
type: string
|
||||
asset_type_name:
|
||||
description: 资产类型名称
|
||||
type: string
|
||||
created_at:
|
||||
description: 创建时间
|
||||
format: date-time
|
||||
type: string
|
||||
from_owner_id:
|
||||
description: 来源所有者ID
|
||||
minimum: 0
|
||||
nullable: true
|
||||
type: integer
|
||||
from_owner_name:
|
||||
description: 来源所有者名称
|
||||
type: string
|
||||
from_owner_type:
|
||||
description: 来源所有者类型
|
||||
type: string
|
||||
id:
|
||||
description: 记录ID
|
||||
minimum: 0
|
||||
type: integer
|
||||
operator_id:
|
||||
description: 操作人ID
|
||||
minimum: 0
|
||||
type: integer
|
||||
operator_name:
|
||||
description: 操作人名称
|
||||
type: string
|
||||
related_card_count:
|
||||
description: 关联卡数量
|
||||
type: integer
|
||||
related_card_ids:
|
||||
description: 关联卡ID列表
|
||||
items:
|
||||
minimum: 0
|
||||
type: integer
|
||||
type: array
|
||||
related_device_id:
|
||||
description: 关联设备ID
|
||||
minimum: 0
|
||||
nullable: true
|
||||
type: integer
|
||||
remark:
|
||||
description: 备注
|
||||
type: string
|
||||
to_owner_id:
|
||||
description: 目标所有者ID
|
||||
minimum: 0
|
||||
type: integer
|
||||
to_owner_name:
|
||||
description: 目标所有者名称
|
||||
type: string
|
||||
to_owner_type:
|
||||
description: 目标所有者类型
|
||||
type: string
|
||||
type: object
|
||||
DtoAssetAllocationRecordResponse:
|
||||
properties:
|
||||
allocation_name:
|
||||
description: 分配类型名称
|
||||
type: string
|
||||
allocation_no:
|
||||
description: 分配单号
|
||||
type: string
|
||||
allocation_type:
|
||||
description: 分配类型 (allocate:分配, recall:回收)
|
||||
type: string
|
||||
asset_id:
|
||||
description: 资产ID
|
||||
minimum: 0
|
||||
type: integer
|
||||
asset_identifier:
|
||||
description: 资产标识符(ICCID或设备号)
|
||||
type: string
|
||||
asset_type:
|
||||
description: 资产类型 (iot_card:物联网卡, device:设备)
|
||||
type: string
|
||||
asset_type_name:
|
||||
description: 资产类型名称
|
||||
type: string
|
||||
created_at:
|
||||
description: 创建时间
|
||||
format: date-time
|
||||
type: string
|
||||
from_owner_id:
|
||||
description: 来源所有者ID
|
||||
minimum: 0
|
||||
nullable: true
|
||||
type: integer
|
||||
from_owner_name:
|
||||
description: 来源所有者名称
|
||||
type: string
|
||||
from_owner_type:
|
||||
description: 来源所有者类型
|
||||
type: string
|
||||
id:
|
||||
description: 记录ID
|
||||
minimum: 0
|
||||
type: integer
|
||||
operator_id:
|
||||
description: 操作人ID
|
||||
minimum: 0
|
||||
type: integer
|
||||
operator_name:
|
||||
description: 操作人名称
|
||||
type: string
|
||||
related_card_count:
|
||||
description: 关联卡数量
|
||||
type: integer
|
||||
related_device_id:
|
||||
description: 关联设备ID
|
||||
minimum: 0
|
||||
nullable: true
|
||||
type: integer
|
||||
remark:
|
||||
description: 备注
|
||||
type: string
|
||||
to_owner_id:
|
||||
description: 目标所有者ID
|
||||
minimum: 0
|
||||
type: integer
|
||||
to_owner_name:
|
||||
description: 目标所有者名称
|
||||
type: string
|
||||
to_owner_type:
|
||||
description: 目标所有者类型
|
||||
type: string
|
||||
type: object
|
||||
DtoAssignPermissionsParams:
|
||||
properties:
|
||||
perm_ids:
|
||||
@@ -832,6 +1062,55 @@ components:
|
||||
description: 失败原因
|
||||
type: string
|
||||
type: object
|
||||
DtoGetUploadURLRequest:
|
||||
properties:
|
||||
content_type:
|
||||
description: 文件 MIME 类型(如:text/csv),留空则自动推断
|
||||
maxLength: 100
|
||||
type: string
|
||||
file_name:
|
||||
description: 文件名(如:cards.csv)
|
||||
maxLength: 255
|
||||
minLength: 1
|
||||
type: string
|
||||
purpose:
|
||||
description: 文件用途 (iot_import:ICCID导入, export:数据导出, attachment:附件)
|
||||
type: string
|
||||
required:
|
||||
- file_name
|
||||
- purpose
|
||||
type: object
|
||||
DtoGetUploadURLResponse:
|
||||
properties:
|
||||
expires_in:
|
||||
description: URL 有效期(秒)
|
||||
type: integer
|
||||
file_key:
|
||||
description: 文件路径标识,上传成功后用于调用业务接口
|
||||
type: string
|
||||
upload_url:
|
||||
description: 预签名上传 URL,使用 PUT 方法上传文件
|
||||
type: string
|
||||
type: object
|
||||
DtoImportIotCardRequest:
|
||||
properties:
|
||||
batch_no:
|
||||
description: 批次号
|
||||
maxLength: 100
|
||||
type: string
|
||||
carrier_id:
|
||||
description: 运营商ID
|
||||
minimum: 1
|
||||
type: integer
|
||||
file_key:
|
||||
description: 对象存储文件路径(通过 /storage/upload-url 获取)
|
||||
maxLength: 500
|
||||
minLength: 1
|
||||
type: string
|
||||
required:
|
||||
- carrier_id
|
||||
- file_key
|
||||
type: object
|
||||
DtoImportIotCardResponse:
|
||||
properties:
|
||||
message:
|
||||
@@ -853,6 +1132,9 @@ components:
|
||||
line:
|
||||
description: 行号
|
||||
type: integer
|
||||
msisdn:
|
||||
description: 接入号
|
||||
type: string
|
||||
reason:
|
||||
description: 原因
|
||||
type: string
|
||||
@@ -985,6 +1267,27 @@ components:
|
||||
description: 总数
|
||||
type: integer
|
||||
type: object
|
||||
DtoListAssetAllocationRecordResponse:
|
||||
properties:
|
||||
list:
|
||||
description: 分配记录列表
|
||||
items:
|
||||
$ref: '#/components/schemas/DtoAssetAllocationRecordResponse'
|
||||
nullable: true
|
||||
type: array
|
||||
page:
|
||||
description: 当前页码
|
||||
type: integer
|
||||
page_size:
|
||||
description: 每页数量
|
||||
type: integer
|
||||
total:
|
||||
description: 总数
|
||||
type: integer
|
||||
total_pages:
|
||||
description: 总页数
|
||||
type: integer
|
||||
type: object
|
||||
DtoListImportTaskResponse:
|
||||
properties:
|
||||
list:
|
||||
@@ -1261,6 +1564,71 @@ components:
|
||||
description: 成功数量
|
||||
type: integer
|
||||
type: object
|
||||
DtoRecallStandaloneCardsRequest:
|
||||
properties:
|
||||
batch_no:
|
||||
description: 批次号(selection_type=filter时可选)
|
||||
maxLength: 100
|
||||
type: string
|
||||
carrier_id:
|
||||
description: 运营商ID(selection_type=filter时可选)
|
||||
minimum: 0
|
||||
nullable: true
|
||||
type: integer
|
||||
from_shop_id:
|
||||
description: 来源店铺ID(被回收方)
|
||||
minimum: 1
|
||||
type: integer
|
||||
iccid_end:
|
||||
description: 结束ICCID(selection_type=range时必填)
|
||||
maxLength: 20
|
||||
type: string
|
||||
iccid_start:
|
||||
description: 起始ICCID(selection_type=range时必填)
|
||||
maxLength: 20
|
||||
type: string
|
||||
iccids:
|
||||
description: ICCID列表(selection_type=list时必填,最多1000个)
|
||||
items:
|
||||
type: string
|
||||
nullable: true
|
||||
type: array
|
||||
remark:
|
||||
description: 备注
|
||||
maxLength: 500
|
||||
type: string
|
||||
selection_type:
|
||||
description: 选卡方式 (list:ICCID列表, range:号段范围, filter:筛选条件)
|
||||
enum:
|
||||
- list
|
||||
- range
|
||||
- filter
|
||||
type: string
|
||||
required:
|
||||
- from_shop_id
|
||||
- selection_type
|
||||
type: object
|
||||
DtoRecallStandaloneCardsResponse:
|
||||
properties:
|
||||
allocation_no:
|
||||
description: 回收单号
|
||||
type: string
|
||||
fail_count:
|
||||
description: 失败数
|
||||
type: integer
|
||||
failed_items:
|
||||
description: 失败项列表
|
||||
items:
|
||||
$ref: '#/components/schemas/DtoAllocationFailedItem'
|
||||
nullable: true
|
||||
type: array
|
||||
success_count:
|
||||
description: 成功数
|
||||
type: integer
|
||||
total_count:
|
||||
description: 待回收总数
|
||||
type: integer
|
||||
type: object
|
||||
DtoRecalledDevice:
|
||||
properties:
|
||||
card_count:
|
||||
@@ -2297,19 +2665,6 @@ components:
|
||||
- message
|
||||
- timestamp
|
||||
type: object
|
||||
FormDataDtoImportIotCardRequest:
|
||||
properties:
|
||||
batch_no:
|
||||
description: 批次号
|
||||
maxLength: 100
|
||||
type: string
|
||||
carrier_id:
|
||||
description: 运营商ID
|
||||
minimum: 1
|
||||
type: integer
|
||||
required:
|
||||
- carrier_id
|
||||
type: object
|
||||
ModelPermission:
|
||||
properties:
|
||||
available_for_role_types:
|
||||
@@ -2779,6 +3134,179 @@ paths:
|
||||
summary: 分配角色
|
||||
tags:
|
||||
- 账号相关
|
||||
/api/admin/asset-allocation-records:
|
||||
get:
|
||||
parameters:
|
||||
- description: 页码
|
||||
in: query
|
||||
name: page
|
||||
schema:
|
||||
description: 页码
|
||||
minimum: 1
|
||||
type: integer
|
||||
- description: 每页数量
|
||||
in: query
|
||||
name: page_size
|
||||
schema:
|
||||
description: 每页数量
|
||||
maximum: 100
|
||||
minimum: 1
|
||||
type: integer
|
||||
- description: 分配类型 (allocate:分配, recall:回收)
|
||||
in: query
|
||||
name: allocation_type
|
||||
schema:
|
||||
description: 分配类型 (allocate:分配, recall:回收)
|
||||
enum:
|
||||
- allocate
|
||||
- recall
|
||||
type: string
|
||||
- description: 资产类型 (iot_card:物联网卡, device:设备)
|
||||
in: query
|
||||
name: asset_type
|
||||
schema:
|
||||
description: 资产类型 (iot_card:物联网卡, device:设备)
|
||||
enum:
|
||||
- iot_card
|
||||
- device
|
||||
type: string
|
||||
- description: 资产标识符(ICCID或设备号,模糊查询)
|
||||
in: query
|
||||
name: asset_identifier
|
||||
schema:
|
||||
description: 资产标识符(ICCID或设备号,模糊查询)
|
||||
maxLength: 50
|
||||
type: string
|
||||
- description: 分配单号(精确匹配)
|
||||
in: query
|
||||
name: allocation_no
|
||||
schema:
|
||||
description: 分配单号(精确匹配)
|
||||
maxLength: 50
|
||||
type: string
|
||||
- description: 来源店铺ID
|
||||
in: query
|
||||
name: from_shop_id
|
||||
schema:
|
||||
description: 来源店铺ID
|
||||
minimum: 0
|
||||
nullable: true
|
||||
type: integer
|
||||
- description: 目标店铺ID
|
||||
in: query
|
||||
name: to_shop_id
|
||||
schema:
|
||||
description: 目标店铺ID
|
||||
minimum: 0
|
||||
nullable: true
|
||||
type: integer
|
||||
- description: 操作人ID
|
||||
in: query
|
||||
name: operator_id
|
||||
schema:
|
||||
description: 操作人ID
|
||||
minimum: 0
|
||||
nullable: true
|
||||
type: integer
|
||||
- description: 创建时间起始
|
||||
in: query
|
||||
name: created_at_start
|
||||
schema:
|
||||
description: 创建时间起始
|
||||
format: date-time
|
||||
nullable: true
|
||||
type: string
|
||||
- description: 创建时间结束
|
||||
in: query
|
||||
name: created_at_end
|
||||
schema:
|
||||
description: 创建时间结束
|
||||
format: date-time
|
||||
nullable: true
|
||||
type: string
|
||||
responses:
|
||||
"200":
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: '#/components/schemas/DtoListAssetAllocationRecordResponse'
|
||||
description: OK
|
||||
"400":
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: '#/components/schemas/ErrorResponse'
|
||||
description: 请求参数错误
|
||||
"401":
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: '#/components/schemas/ErrorResponse'
|
||||
description: 未认证或认证已过期
|
||||
"403":
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: '#/components/schemas/ErrorResponse'
|
||||
description: 无权访问
|
||||
"500":
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: '#/components/schemas/ErrorResponse'
|
||||
description: 服务器内部错误
|
||||
security:
|
||||
- BearerAuth: []
|
||||
summary: 分配记录列表
|
||||
tags:
|
||||
- 资产分配记录
|
||||
/api/admin/asset-allocation-records/{id}:
|
||||
get:
|
||||
parameters:
|
||||
- description: 记录ID
|
||||
in: path
|
||||
name: id
|
||||
required: true
|
||||
schema:
|
||||
description: 记录ID
|
||||
minimum: 1
|
||||
type: integer
|
||||
responses:
|
||||
"200":
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: '#/components/schemas/DtoAssetAllocationRecordDetailResponse'
|
||||
description: OK
|
||||
"400":
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: '#/components/schemas/ErrorResponse'
|
||||
description: 请求参数错误
|
||||
"401":
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: '#/components/schemas/ErrorResponse'
|
||||
description: 未认证或认证已过期
|
||||
"403":
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: '#/components/schemas/ErrorResponse'
|
||||
description: 无权访问
|
||||
"500":
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: '#/components/schemas/ErrorResponse'
|
||||
description: 服务器内部错误
|
||||
security:
|
||||
- BearerAuth: []
|
||||
summary: 分配记录详情
|
||||
tags:
|
||||
- 资产分配记录
|
||||
/api/admin/commission/withdrawal-requests:
|
||||
get:
|
||||
parameters:
|
||||
@@ -4006,27 +4534,37 @@ paths:
|
||||
- 企业客户管理
|
||||
/api/admin/iot-cards/import:
|
||||
post:
|
||||
parameters:
|
||||
- description: 运营商ID
|
||||
in: query
|
||||
name: carrier_id
|
||||
required: true
|
||||
schema:
|
||||
description: 运营商ID
|
||||
minimum: 1
|
||||
type: integer
|
||||
- description: 批次号
|
||||
in: query
|
||||
name: batch_no
|
||||
schema:
|
||||
description: 批次号
|
||||
maxLength: 100
|
||||
type: string
|
||||
description: |-
|
||||
## ⚠️ 接口变更说明(BREAKING CHANGE)
|
||||
|
||||
本接口已从 `multipart/form-data` 改为 `application/json`。
|
||||
|
||||
### 完整导入流程
|
||||
|
||||
1. **获取上传 URL**: 调用 `POST /api/admin/storage/upload-url`
|
||||
2. **上传 CSV 文件**: 使用预签名 URL 上传文件到对象存储
|
||||
3. **调用本接口**: 使用返回的 `file_key` 提交导入任务
|
||||
|
||||
### 请求示例
|
||||
|
||||
```json
|
||||
{
|
||||
"carrier_id": 1,
|
||||
"batch_no": "BATCH-2025-01",
|
||||
"file_key": "imports/2025/01/24/abc123.csv"
|
||||
}
|
||||
```
|
||||
|
||||
### CSV 文件格式
|
||||
|
||||
- 必须包含两列:`iccid`, `msisdn`
|
||||
- 首行为表头
|
||||
- 编码:UTF-8
|
||||
requestBody:
|
||||
content:
|
||||
application/x-www-form-urlencoded:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: '#/components/schemas/FormDataDtoImportIotCardRequest'
|
||||
$ref: '#/components/schemas/DtoImportIotCardRequest'
|
||||
responses:
|
||||
"200":
|
||||
content:
|
||||
@@ -4060,7 +4598,7 @@ paths:
|
||||
description: 服务器内部错误
|
||||
security:
|
||||
- BearerAuth: []
|
||||
summary: 批量导入ICCID
|
||||
summary: 批量导入IoT卡(ICCID+MSISDN)
|
||||
tags:
|
||||
- IoT卡管理
|
||||
/api/admin/iot-cards/import-tasks:
|
||||
@@ -4340,6 +4878,92 @@ paths:
|
||||
summary: 单卡列表(未绑定设备)
|
||||
tags:
|
||||
- IoT卡管理
|
||||
/api/admin/iot-cards/standalone/allocate:
|
||||
post:
|
||||
requestBody:
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: '#/components/schemas/DtoAllocateStandaloneCardsRequest'
|
||||
responses:
|
||||
"200":
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: '#/components/schemas/DtoAllocateStandaloneCardsResponse'
|
||||
description: OK
|
||||
"400":
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: '#/components/schemas/ErrorResponse'
|
||||
description: 请求参数错误
|
||||
"401":
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: '#/components/schemas/ErrorResponse'
|
||||
description: 未认证或认证已过期
|
||||
"403":
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: '#/components/schemas/ErrorResponse'
|
||||
description: 无权访问
|
||||
"500":
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: '#/components/schemas/ErrorResponse'
|
||||
description: 服务器内部错误
|
||||
security:
|
||||
- BearerAuth: []
|
||||
summary: 批量分配单卡
|
||||
tags:
|
||||
- IoT卡管理
|
||||
/api/admin/iot-cards/standalone/recall:
|
||||
post:
|
||||
requestBody:
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: '#/components/schemas/DtoRecallStandaloneCardsRequest'
|
||||
responses:
|
||||
"200":
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: '#/components/schemas/DtoRecallStandaloneCardsResponse'
|
||||
description: OK
|
||||
"400":
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: '#/components/schemas/ErrorResponse'
|
||||
description: 请求参数错误
|
||||
"401":
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: '#/components/schemas/ErrorResponse'
|
||||
description: 未认证或认证已过期
|
||||
"403":
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: '#/components/schemas/ErrorResponse'
|
||||
description: 无权访问
|
||||
"500":
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: '#/components/schemas/ErrorResponse'
|
||||
description: 服务器内部错误
|
||||
security:
|
||||
- BearerAuth: []
|
||||
summary: 批量回收单卡
|
||||
tags:
|
||||
- IoT卡管理
|
||||
/api/admin/login:
|
||||
post:
|
||||
requestBody:
|
||||
@@ -6740,6 +7364,99 @@ paths:
|
||||
summary: 代理商佣金列表
|
||||
tags:
|
||||
- 代理商佣金管理
|
||||
/api/admin/storage/upload-url:
|
||||
post:
|
||||
description: |-
|
||||
## 文件上传流程
|
||||
|
||||
本接口用于获取对象存储的预签名上传 URL,实现前端直传文件到对象存储。
|
||||
|
||||
### 完整流程
|
||||
|
||||
1. **调用本接口** 获取预签名 URL 和 file_key
|
||||
2. **使用预签名 URL 上传文件** 发起 PUT 请求直接上传到对象存储
|
||||
3. **调用业务接口** 使用 file_key 调用相关业务接口(如 ICCID 导入)
|
||||
|
||||
### 前端上传示例
|
||||
|
||||
```javascript
|
||||
// 1. 获取预签名 URL
|
||||
const { data } = await api.post('/storage/upload-url', {
|
||||
file_name: 'cards.csv',
|
||||
content_type: 'text/csv',
|
||||
purpose: 'iot_import'
|
||||
});
|
||||
|
||||
// 2. 上传文件到对象存储
|
||||
await fetch(data.upload_url, {
|
||||
method: 'PUT',
|
||||
headers: { 'Content-Type': 'text/csv' },
|
||||
body: file
|
||||
});
|
||||
|
||||
// 3. 调用业务接口
|
||||
await api.post('/iot-cards/import', {
|
||||
carrier_id: 1,
|
||||
batch_no: 'BATCH-2025-01',
|
||||
file_key: data.file_key
|
||||
});
|
||||
```
|
||||
|
||||
### purpose 可选值
|
||||
|
||||
| 值 | 说明 | 生成路径格式 |
|
||||
|---|------|-------------|
|
||||
| iot_import | ICCID 导入 | imports/YYYY/MM/DD/uuid.csv |
|
||||
| export | 数据导出 | exports/YYYY/MM/DD/uuid.xlsx |
|
||||
| attachment | 附件上传 | attachments/YYYY/MM/DD/uuid.ext |
|
||||
|
||||
### 注意事项
|
||||
|
||||
- 预签名 URL 有效期 **15 分钟**,请及时使用
|
||||
- 上传时 Content-Type 需与请求时一致
|
||||
- file_key 在上传成功后永久有效,用于后续业务接口调用
|
||||
- 上传失败时可重新调用本接口获取新的 URL
|
||||
requestBody:
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: '#/components/schemas/DtoGetUploadURLRequest'
|
||||
responses:
|
||||
"200":
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: '#/components/schemas/DtoGetUploadURLResponse'
|
||||
description: OK
|
||||
"400":
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: '#/components/schemas/ErrorResponse'
|
||||
description: 请求参数错误
|
||||
"401":
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: '#/components/schemas/ErrorResponse'
|
||||
description: 未认证或认证已过期
|
||||
"403":
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: '#/components/schemas/ErrorResponse'
|
||||
description: 无权访问
|
||||
"500":
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: '#/components/schemas/ErrorResponse'
|
||||
description: 服务器内部错误
|
||||
security:
|
||||
- BearerAuth: []
|
||||
summary: 获取文件上传预签名 URL
|
||||
tags:
|
||||
- 对象存储
|
||||
/api/admin/tasks/{id}:
|
||||
get:
|
||||
parameters:
|
||||
|
||||
@@ -173,15 +173,65 @@ func registerXxxRoutes(
|
||||
|
||||
```go
|
||||
type RouteSpec struct {
|
||||
Summary string // 操作摘要(中文,简短)
|
||||
Input interface{} // 请求参数 DTO
|
||||
Output interface{} // 响应结果 DTO
|
||||
Tags []string // 分类标签(用于文档分组)
|
||||
Auth bool // 是否需要认证
|
||||
Summary string // 操作摘要(中文,简短,一行)
|
||||
Description string // 详细说明,支持 Markdown 语法(可选)
|
||||
Input interface{} // 请求参数 DTO
|
||||
Output interface{} // 响应结果 DTO
|
||||
Tags []string // 分类标签(用于文档分组)
|
||||
Auth bool // 是否需要认证
|
||||
}
|
||||
```
|
||||
|
||||
### 4. 完整示例
|
||||
### 4. Description 字段(Markdown 说明)
|
||||
|
||||
`Description` 字段用于添加接口的详细说明,支持 **CommonMark Markdown** 语法。Apifox 等 OpenAPI 工具会正确渲染这些 Markdown 内容。
|
||||
|
||||
**使用场景**:
|
||||
- 业务规则说明
|
||||
- 请求频率限制
|
||||
- 注意事项
|
||||
- 错误码说明
|
||||
- 数据格式说明
|
||||
|
||||
**示例**:
|
||||
```go
|
||||
Register(router, doc, basePath, "POST", "/login", handler.Login, RouteSpec{
|
||||
Summary: "后台登录",
|
||||
Description: `## 登录说明
|
||||
|
||||
**请求频率限制**:每分钟最多 10 次
|
||||
|
||||
### 注意事项
|
||||
1. 密码错误 5 次后账号将被锁定 30 分钟
|
||||
2. Token 有效期为 24 小时
|
||||
|
||||
### 返回码说明
|
||||
| 错误码 | 说明 |
|
||||
|--------|------|
|
||||
| 1001 | 用户名或密码错误 |
|
||||
| 1002 | 账号已被锁定 |
|
||||
`,
|
||||
Tags: []string{"认证"},
|
||||
Input: new(dto.LoginRequest),
|
||||
Output: new(dto.LoginResponse),
|
||||
Auth: false,
|
||||
})
|
||||
```
|
||||
|
||||
**支持的 Markdown 语法**:
|
||||
- 标题:`#`、`##`、`###`
|
||||
- 列表:`-`、`1.`
|
||||
- 表格:`| 列1 | 列2 |`
|
||||
- 代码:`` `code` `` 和 ` ```code block``` `
|
||||
- 强调:`**粗体**`、`*斜体*`
|
||||
- 链接:`[文本](url)`
|
||||
|
||||
**最佳实践**:
|
||||
- 保持简洁,控制在 500 字以内
|
||||
- 使用结构化的 Markdown(标题、列表、表格)提高可读性
|
||||
- 避免使用 HTML 标签(兼容性较差)
|
||||
|
||||
### 5. 完整示例
|
||||
|
||||
```go
|
||||
func registerShopRoutes(router fiber.Router, handler *admin.ShopHandler, doc *openapi.Generator, basePath string) {
|
||||
|
||||
@@ -30,16 +30,18 @@
|
||||
|
||||
### 配置项
|
||||
|
||||
在 `configs/config.yaml` 中配置 Token 有效期:
|
||||
通过环境变量配置 Token 有效期:
|
||||
|
||||
```yaml
|
||||
jwt:
|
||||
secret_key: "your-secret-key-here"
|
||||
token_duration: 3600 # JWT 有效期(个人客户,秒)
|
||||
access_token_ttl: 86400 # Access Token 有效期(B端,秒)
|
||||
refresh_token_ttl: 604800 # Refresh Token 有效期(B端,秒)
|
||||
```bash
|
||||
# JWT 配置
|
||||
export JUNHONG_JWT_SECRET_KEY="your-secret-key-here"
|
||||
export JUNHONG_JWT_TOKEN_DURATION="24h" # JWT 有效期(个人客户)
|
||||
export JUNHONG_JWT_ACCESS_TOKEN_TTL="24h" # Access Token 有效期(B端)
|
||||
export JUNHONG_JWT_REFRESH_TOKEN_TTL="168h" # Refresh Token 有效期(B端,7天)
|
||||
```
|
||||
|
||||
详细配置说明见 [环境变量配置文档](environment-variables.md)
|
||||
|
||||
---
|
||||
|
||||
## 在路由中集成认证
|
||||
|
||||
@@ -44,7 +44,7 @@
|
||||
│ │ │
|
||||
│ ┌───────────▼─────────────────────┐ │
|
||||
│ │ 服务运行中 │ │
|
||||
│ │ - API: 0.0.0.0:8088 │ │
|
||||
│ │ - API: 0.0.0.0:3000 │ │
|
||||
│ │ - Worker: 后台任务处理 │ │
|
||||
│ └──────────────────────────────────┘ │
|
||||
└───────────────────────────────────────┘
|
||||
@@ -98,81 +98,34 @@ docker ps | grep runner
|
||||
mkdir -p /home/qycard001/app/junhong_cmp
|
||||
cd /home/qycard001/app/junhong_cmp
|
||||
|
||||
# 创建必要的子目录
|
||||
mkdir -p configs logs
|
||||
# 创建日志目录(配置已嵌入二进制文件,无需 configs 目录)
|
||||
mkdir -p logs
|
||||
```
|
||||
|
||||
### 1.3 准备配置文件
|
||||
### 1.3 配置说明
|
||||
|
||||
#### 创建 `.env` 文件(数据库迁移配置)
|
||||
系统使用**嵌入式配置 + 环境变量覆盖**机制:
|
||||
|
||||
- 默认配置已编译在二进制文件中
|
||||
- 通过 `docker-compose.prod.yml` 中的环境变量覆盖配置
|
||||
- 环境变量前缀:`JUNHONG_`
|
||||
- 格式:`JUNHONG_{配置路径}`,路径分隔符用下划线替代点号
|
||||
|
||||
**无需手动创建配置文件**,所有配置在 `docker-compose.prod.yml` 的 `environment` 中管理。
|
||||
|
||||
### 1.4 部署文件
|
||||
|
||||
`docker-compose.prod.yml` 由 CI/CD 自动从代码仓库复制到部署目录,无需手动操作。
|
||||
|
||||
如需手动部署,可从代码仓库复制:
|
||||
|
||||
```bash
|
||||
cat > /home/qycard001/app/junhong_cmp/.env << 'EOF'
|
||||
MIGRATIONS_DIR=migrations
|
||||
DB_HOST=cxd.whcxd.cn
|
||||
DB_PORT=16159
|
||||
DB_USER=erp_pgsql
|
||||
DB_PASSWORD=erp_2025
|
||||
DB_NAME=junhong_cmp_test
|
||||
DB_SSLMODE=disable
|
||||
EOF
|
||||
```
|
||||
|
||||
#### 创建 `configs/config.yaml`(应用配置)
|
||||
|
||||
```bash
|
||||
cat > /home/qycard001/app/junhong_cmp/configs/config.yaml << 'EOF'
|
||||
server:
|
||||
port: 8088
|
||||
read_timeout: 60
|
||||
write_timeout: 60
|
||||
|
||||
database:
|
||||
host: cxd.whcxd.cn
|
||||
port: 16159
|
||||
user: erp_pgsql
|
||||
password: erp_2025
|
||||
dbname: junhong_cmp_test
|
||||
sslmode: disable
|
||||
max_open_conns: 100
|
||||
max_idle_conns: 10
|
||||
|
||||
redis:
|
||||
host: 你的Redis地址
|
||||
port: 6379
|
||||
password: ""
|
||||
db: 0
|
||||
|
||||
logging:
|
||||
level: info
|
||||
output: logs/app.log
|
||||
max_size: 100
|
||||
max_backups: 7
|
||||
max_age: 30
|
||||
compress: true
|
||||
|
||||
middleware:
|
||||
enable_rate_limiter: false
|
||||
EOF
|
||||
```
|
||||
|
||||
**重要**:将 `你的Redis地址` 替换为实际的 Redis 地址。
|
||||
|
||||
### 1.4 复制部署文件
|
||||
|
||||
从代码仓库复制 `docker-compose.prod.yml` 到服务器:
|
||||
|
||||
```bash
|
||||
# 在服务器上执行
|
||||
cd /home/qycard001/app/junhong_cmp
|
||||
|
||||
# 方式1: 使用 Git(推荐)
|
||||
# 方式1: 使用 Git
|
||||
git clone <你的仓库地址> temp
|
||||
cp temp/docker-compose.prod.yml ./docker-compose.prod.yml
|
||||
rm -rf temp
|
||||
|
||||
# 方式2: 从本地上传
|
||||
# 在本地执行:
|
||||
# scp -P 52022 docker-compose.prod.yml qycard001@47.111.166.169:/home/qycard001/app/junhong_cmp/
|
||||
```
|
||||
|
||||
@@ -251,7 +204,7 @@ docker-compose -f docker-compose.prod.yml logs -f
|
||||
|
||||
```bash
|
||||
# 测试 API 健康检查
|
||||
curl http://localhost:8088/health
|
||||
curl http://localhost:3000/health
|
||||
|
||||
# 预期输出:
|
||||
# {"code":0,"msg":"ok","data":{"status":"healthy"},"timestamp":1234567890}
|
||||
@@ -397,9 +350,9 @@ docker system prune -a -f --volumes
|
||||
|
||||
**排查步骤**:
|
||||
1. 查看容器日志:`docker-compose -f docker-compose.prod.yml logs api`
|
||||
2. 检查配置文件是否正确(数据库连接、Redis 连接)
|
||||
2. 检查 `docker-compose.prod.yml` 中的环境变量配置是否正确(数据库连接、Redis 连接)
|
||||
3. 确认外部依赖(PostgreSQL、Redis)是否可访问
|
||||
4. 手动测试健康检查:`curl http://localhost:8088/health`
|
||||
4. 手动测试健康检查:`curl http://localhost:3000/health`
|
||||
|
||||
### Q2: 数据库迁移失败
|
||||
|
||||
@@ -470,7 +423,7 @@ docker restart docker-runner-01
|
||||
```bash
|
||||
# 仅开放必要端口
|
||||
sudo ufw allow 52022/tcp # SSH
|
||||
sudo ufw allow 8088/tcp # API(如果需要外部访问)
|
||||
sudo ufw allow 3000/tcp # API(如果需要外部访问)
|
||||
sudo ufw enable
|
||||
```
|
||||
|
||||
@@ -480,7 +433,7 @@ docker restart docker-runner-01
|
||||
|
||||
4. **备份策略**:
|
||||
- 定期备份数据库
|
||||
- 定期备份配置文件(`.env`、`config.yaml`)
|
||||
- 定期备份 `docker-compose.prod.yml`(包含所有配置)
|
||||
|
||||
---
|
||||
|
||||
|
||||
212
docs/environment-variables.md
Normal file
212
docs/environment-variables.md
Normal file
@@ -0,0 +1,212 @@
|
||||
# 环境变量配置文档
|
||||
|
||||
## 概述
|
||||
|
||||
君鸿卡管系统使用嵌入式配置机制,默认配置编译在二进制文件中,通过环境变量进行覆盖。
|
||||
|
||||
**环境变量前缀**: `JUNHONG_`
|
||||
**格式规则**: 配置路径中的 `.` 替换为 `_`,全部大写
|
||||
|
||||
## 必填配置
|
||||
|
||||
以下配置没有合理的默认值,必须通过环境变量设置:
|
||||
|
||||
### 数据库配置
|
||||
|
||||
| 环境变量 | 说明 | 示例 |
|
||||
|---------|------|------|
|
||||
| `JUNHONG_DATABASE_HOST` | 数据库主机地址 | `localhost` |
|
||||
| `JUNHONG_DATABASE_PORT` | 数据库端口 | `5432` |
|
||||
| `JUNHONG_DATABASE_USER` | 数据库用户名 | `postgres` |
|
||||
| `JUNHONG_DATABASE_PASSWORD` | 数据库密码 | `your_password` |
|
||||
| `JUNHONG_DATABASE_DBNAME` | 数据库名称 | `junhong_cmp` |
|
||||
|
||||
### Redis 配置
|
||||
|
||||
| 环境变量 | 说明 | 示例 |
|
||||
|---------|------|------|
|
||||
| `JUNHONG_REDIS_ADDRESS` | Redis 主机地址 | `localhost` |
|
||||
|
||||
### JWT 配置
|
||||
|
||||
| 环境变量 | 说明 | 示例 |
|
||||
|---------|------|------|
|
||||
| `JUNHONG_JWT_SECRET_KEY` | JWT 签名密钥(生产环境必须修改) | `your-secret-key` |
|
||||
|
||||
## 可选配置
|
||||
|
||||
以下配置有合理的默认值,可按需覆盖:
|
||||
|
||||
### 服务器配置
|
||||
|
||||
| 环境变量 | 默认值 | 说明 |
|
||||
|---------|--------|------|
|
||||
| `JUNHONG_SERVER_ADDRESS` | `:3000` | 服务监听地址 |
|
||||
| `JUNHONG_SERVER_READ_TIMEOUT` | `30s` | 读取超时时间 |
|
||||
| `JUNHONG_SERVER_WRITE_TIMEOUT` | `30s` | 写入超时时间 |
|
||||
| `JUNHONG_SERVER_SHUTDOWN_TIMEOUT` | `30s` | 优雅关闭超时 |
|
||||
| `JUNHONG_SERVER_PREFORK` | `false` | 是否启用预分叉模式 |
|
||||
|
||||
### 数据库连接池
|
||||
|
||||
| 环境变量 | 默认值 | 说明 |
|
||||
|---------|--------|------|
|
||||
| `JUNHONG_DATABASE_SSLMODE` | `disable` | SSL 模式 |
|
||||
| `JUNHONG_DATABASE_MAX_OPEN_CONNS` | `25` | 最大打开连接数 |
|
||||
| `JUNHONG_DATABASE_MAX_IDLE_CONNS` | `10` | 最大空闲连接数 |
|
||||
| `JUNHONG_DATABASE_CONN_MAX_LIFETIME` | `1h` | 连接最大生命周期 |
|
||||
|
||||
### Redis 配置
|
||||
|
||||
| 环境变量 | 默认值 | 说明 |
|
||||
|---------|--------|------|
|
||||
| `JUNHONG_REDIS_PORT` | `6379` | Redis 端口 |
|
||||
| `JUNHONG_REDIS_PASSWORD` | `""` | Redis 密码 |
|
||||
| `JUNHONG_REDIS_DB` | `0` | Redis 数据库编号 |
|
||||
| `JUNHONG_REDIS_POOL_SIZE` | `100` | 连接池大小 |
|
||||
| `JUNHONG_REDIS_MIN_IDLE_CONNS` | `10` | 最小空闲连接数 |
|
||||
| `JUNHONG_REDIS_DIAL_TIMEOUT` | `5s` | 连接超时 |
|
||||
| `JUNHONG_REDIS_READ_TIMEOUT` | `3s` | 读取超时 |
|
||||
| `JUNHONG_REDIS_WRITE_TIMEOUT` | `3s` | 写入超时 |
|
||||
|
||||
### 日志配置
|
||||
|
||||
| 环境变量 | 默认值 | 说明 |
|
||||
|---------|--------|------|
|
||||
| `JUNHONG_LOGGING_LEVEL` | `info` | 日志级别 (debug/info/warn/error) |
|
||||
| `JUNHONG_LOGGING_DEVELOPMENT` | `false` | 开发模式(启用彩色输出) |
|
||||
| `JUNHONG_LOGGING_APP_LOG_FILENAME` | `logs/app.log` | 应用日志文件路径 |
|
||||
| `JUNHONG_LOGGING_APP_LOG_MAX_SIZE` | `100` | 日志文件最大大小 (MB) |
|
||||
| `JUNHONG_LOGGING_APP_LOG_MAX_BACKUPS` | `7` | 最大备份文件数 |
|
||||
| `JUNHONG_LOGGING_APP_LOG_MAX_AGE` | `30` | 日志保留天数 |
|
||||
| `JUNHONG_LOGGING_APP_LOG_COMPRESS` | `true` | 是否压缩旧日志 |
|
||||
| `JUNHONG_LOGGING_ACCESS_LOG_FILENAME` | `logs/access.log` | 访问日志文件路径 |
|
||||
|
||||
### JWT 配置
|
||||
|
||||
| 环境变量 | 默认值 | 说明 |
|
||||
|---------|--------|------|
|
||||
| `JUNHONG_JWT_TOKEN_DURATION` | `24h` | Token 有效期 |
|
||||
| `JUNHONG_JWT_ACCESS_TOKEN_TTL` | `24h` | Access Token TTL |
|
||||
| `JUNHONG_JWT_REFRESH_TOKEN_TTL` | `168h` | Refresh Token TTL (7天) |
|
||||
|
||||
### 队列配置
|
||||
|
||||
| 环境变量 | 默认值 | 说明 |
|
||||
|---------|--------|------|
|
||||
| `JUNHONG_QUEUE_CONCURRENCY` | `10` | 并发 Worker 数量 |
|
||||
| `JUNHONG_QUEUE_RETRY_MAX` | `3` | 最大重试次数 |
|
||||
| `JUNHONG_QUEUE_TIMEOUT` | `30m` | 任务超时时间 |
|
||||
|
||||
### 限流中间件
|
||||
|
||||
| 环境变量 | 默认值 | 说明 |
|
||||
|---------|--------|------|
|
||||
| `JUNHONG_MIDDLEWARE_ENABLE_RATE_LIMITER` | `false` | 启用限流 |
|
||||
| `JUNHONG_MIDDLEWARE_RATE_LIMITER_MAX` | `100` | 最大请求数 |
|
||||
| `JUNHONG_MIDDLEWARE_RATE_LIMITER_EXPIRATION` | `1m` | 时间窗口 |
|
||||
| `JUNHONG_MIDDLEWARE_RATE_LIMITER_STORAGE` | `memory` | 存储后端 (memory/redis) |
|
||||
|
||||
### 对象存储配置
|
||||
|
||||
| 环境变量 | 默认值 | 说明 |
|
||||
|---------|--------|------|
|
||||
| `JUNHONG_STORAGE_PROVIDER` | `""` | 存储提供商 (s3) |
|
||||
| `JUNHONG_STORAGE_TEMP_DIR` | `/tmp/junhong` | 临时文件目录 |
|
||||
| `JUNHONG_STORAGE_S3_ENDPOINT` | `""` | S3 端点 |
|
||||
| `JUNHONG_STORAGE_S3_REGION` | `""` | S3 区域 |
|
||||
| `JUNHONG_STORAGE_S3_BUCKET` | `""` | S3 存储桶 |
|
||||
| `JUNHONG_STORAGE_S3_ACCESS_KEY_ID` | `""` | S3 访问密钥 ID |
|
||||
| `JUNHONG_STORAGE_S3_SECRET_ACCESS_KEY` | `""` | S3 访问密钥 |
|
||||
| `JUNHONG_STORAGE_S3_USE_SSL` | `true` | 是否使用 SSL |
|
||||
| `JUNHONG_STORAGE_S3_PATH_STYLE` | `true` | 是否使用路径风格 |
|
||||
| `JUNHONG_STORAGE_PRESIGN_UPLOAD_EXPIRES` | `1h` | 预签名上传 URL 有效期 |
|
||||
| `JUNHONG_STORAGE_PRESIGN_DOWNLOAD_EXPIRES` | `1h` | 预签名下载 URL 有效期 |
|
||||
|
||||
### 短信配置
|
||||
|
||||
| 环境变量 | 默认值 | 说明 |
|
||||
|---------|--------|------|
|
||||
| `JUNHONG_SMS_GATEWAY_URL` | `""` | 短信网关 URL |
|
||||
| `JUNHONG_SMS_USERNAME` | `""` | 短信账号 |
|
||||
| `JUNHONG_SMS_PASSWORD` | `""` | 短信密码 |
|
||||
| `JUNHONG_SMS_SIGNATURE` | `""` | 短信签名 |
|
||||
| `JUNHONG_SMS_TIMEOUT` | `10s` | 请求超时 |
|
||||
|
||||
### 默认管理员
|
||||
|
||||
| 环境变量 | 默认值 | 说明 |
|
||||
|---------|--------|------|
|
||||
| `JUNHONG_DEFAULT_ADMIN_USERNAME` | `admin` | 默认管理员用户名 |
|
||||
| `JUNHONG_DEFAULT_ADMIN_PASSWORD` | `Admin@123456` | 默认管理员密码 |
|
||||
| `JUNHONG_DEFAULT_ADMIN_PHONE` | `13800000000` | 默认管理员手机号 |
|
||||
|
||||
## Docker Compose 示例
|
||||
|
||||
```yaml
|
||||
version: '3.8'
|
||||
|
||||
services:
|
||||
api:
|
||||
image: registry.boss160.cn/junhong/cmp-fiber-api:latest
|
||||
environment:
|
||||
- JUNHONG_DATABASE_HOST=postgres
|
||||
- JUNHONG_DATABASE_PORT=5432
|
||||
- JUNHONG_DATABASE_USER=junhong
|
||||
- JUNHONG_DATABASE_PASSWORD=secret123
|
||||
- JUNHONG_DATABASE_DBNAME=junhong_cmp
|
||||
- JUNHONG_REDIS_ADDRESS=redis
|
||||
- JUNHONG_JWT_SECRET_KEY=your-production-secret-key
|
||||
- JUNHONG_LOGGING_LEVEL=info
|
||||
volumes:
|
||||
- ./logs:/app/logs
|
||||
ports:
|
||||
- "3000:3000"
|
||||
|
||||
worker:
|
||||
image: registry.boss160.cn/junhong/cmp-fiber-worker:latest
|
||||
environment:
|
||||
- JUNHONG_DATABASE_HOST=postgres
|
||||
- JUNHONG_DATABASE_PORT=5432
|
||||
- JUNHONG_DATABASE_USER=junhong
|
||||
- JUNHONG_DATABASE_PASSWORD=secret123
|
||||
- JUNHONG_DATABASE_DBNAME=junhong_cmp
|
||||
- JUNHONG_REDIS_ADDRESS=redis
|
||||
- JUNHONG_JWT_SECRET_KEY=your-production-secret-key
|
||||
volumes:
|
||||
- ./logs:/app/logs
|
||||
|
||||
postgres:
|
||||
image: postgres:14
|
||||
environment:
|
||||
- POSTGRES_USER=junhong
|
||||
- POSTGRES_PASSWORD=secret123
|
||||
- POSTGRES_DB=junhong_cmp
|
||||
|
||||
redis:
|
||||
image: redis:6
|
||||
```
|
||||
|
||||
## 本地开发
|
||||
|
||||
本地开发可以创建 `.env` 文件(不要提交到 Git):
|
||||
|
||||
```bash
|
||||
# .env
|
||||
JUNHONG_DATABASE_HOST=localhost
|
||||
JUNHONG_DATABASE_PORT=5432
|
||||
JUNHONG_DATABASE_USER=postgres
|
||||
JUNHONG_DATABASE_PASSWORD=postgres
|
||||
JUNHONG_DATABASE_DBNAME=junhong_cmp_dev
|
||||
JUNHONG_REDIS_ADDRESS=localhost
|
||||
JUNHONG_JWT_SECRET_KEY=dev-secret-key
|
||||
JUNHONG_LOGGING_LEVEL=debug
|
||||
JUNHONG_LOGGING_DEVELOPMENT=true
|
||||
```
|
||||
|
||||
然后使用 `source .env` 加载环境变量后运行:
|
||||
|
||||
```bash
|
||||
source .env
|
||||
go run cmd/api/main.go
|
||||
```
|
||||
163
docs/object-storage/使用指南.md
Normal file
163
docs/object-storage/使用指南.md
Normal file
@@ -0,0 +1,163 @@
|
||||
# 对象存储使用指南
|
||||
|
||||
本文档介绍如何在后端代码中使用对象存储服务。
|
||||
|
||||
## 配置
|
||||
|
||||
通过环境变量配置对象存储:
|
||||
|
||||
```bash
|
||||
# 存储提供商
|
||||
export JUNHONG_STORAGE_PROVIDER="s3"
|
||||
|
||||
# S3 配置
|
||||
export JUNHONG_STORAGE_S3_ENDPOINT="http://obs-helf.cucloud.cn"
|
||||
export JUNHONG_STORAGE_S3_REGION="cn-langfang-2"
|
||||
export JUNHONG_STORAGE_S3_BUCKET="cmp"
|
||||
export JUNHONG_STORAGE_S3_ACCESS_KEY_ID="YOUR_ACCESS_KEY"
|
||||
export JUNHONG_STORAGE_S3_SECRET_ACCESS_KEY="YOUR_SECRET_KEY"
|
||||
export JUNHONG_STORAGE_S3_USE_SSL="false"
|
||||
export JUNHONG_STORAGE_S3_PATH_STYLE="true"
|
||||
|
||||
# 预签名 URL 配置
|
||||
export JUNHONG_STORAGE_PRESIGN_UPLOAD_EXPIRES="15m"
|
||||
export JUNHONG_STORAGE_PRESIGN_DOWNLOAD_EXPIRES="24h"
|
||||
|
||||
# 临时文件目录
|
||||
export JUNHONG_STORAGE_TEMP_DIR="/tmp/junhong-storage"
|
||||
```
|
||||
|
||||
详细配置说明见 [环境变量配置文档](../environment-variables.md)
|
||||
|
||||
## StorageService 使用
|
||||
|
||||
### 获取预签名上传 URL
|
||||
|
||||
```go
|
||||
result, err := storageService.GetUploadURL(ctx, "iot_import", "cards.csv", "text/csv")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
// result.URL - 预签名上传 URL
|
||||
// result.FileKey - 文件路径(用于后续业务接口)
|
||||
// result.ExpiresIn - URL 有效期(秒)
|
||||
```
|
||||
|
||||
### 下载文件到临时目录
|
||||
|
||||
```go
|
||||
localPath, cleanup, err := storageService.DownloadToTemp(ctx, fileKey)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer cleanup() // 处理完成后自动删除临时文件
|
||||
|
||||
// 使用 localPath 读取文件内容
|
||||
f, _ := os.Open(localPath)
|
||||
defer f.Close()
|
||||
```
|
||||
|
||||
### 直接上传文件
|
||||
|
||||
```go
|
||||
reader := bytes.NewReader(content)
|
||||
err := storageService.Provider().Upload(ctx, fileKey, reader, "text/csv")
|
||||
```
|
||||
|
||||
### 检查文件是否存在
|
||||
|
||||
```go
|
||||
exists, err := storageService.Provider().Exists(ctx, fileKey)
|
||||
```
|
||||
|
||||
### 删除文件
|
||||
|
||||
```go
|
||||
err := storageService.Provider().Delete(ctx, fileKey)
|
||||
```
|
||||
|
||||
## Purpose 类型
|
||||
|
||||
| Purpose | 说明 | 生成路径 | ContentType |
|
||||
|---------|------|---------|-------------|
|
||||
| iot_import | ICCID 导入 | imports/YYYY/MM/DD/uuid.csv | text/csv |
|
||||
| export | 数据导出 | exports/YYYY/MM/DD/uuid.xlsx | application/vnd.openxmlformats... |
|
||||
| attachment | 附件上传 | attachments/YYYY/MM/DD/uuid.ext | 自动检测 |
|
||||
|
||||
## 错误处理
|
||||
|
||||
存储相关错误码定义在 `pkg/errors/codes.go`:
|
||||
|
||||
| 错误码 | 说明 |
|
||||
|-------|------|
|
||||
| 1090 | 对象存储服务未配置 |
|
||||
| 1091 | 文件上传失败 |
|
||||
| 1092 | 文件下载失败 |
|
||||
| 1093 | 文件不存在 |
|
||||
| 1094 | 不支持的文件用途 |
|
||||
| 1095 | 不支持的文件类型 |
|
||||
|
||||
## 在 Handler 中使用
|
||||
|
||||
```go
|
||||
type MyHandler struct {
|
||||
storageService *storage.Service
|
||||
}
|
||||
|
||||
func (h *MyHandler) Upload(c *fiber.Ctx) error {
|
||||
var req dto.GetUploadURLRequest
|
||||
if err := c.BodyParser(&req); err != nil {
|
||||
return errors.New(errors.CodeInvalidParam, "参数解析失败")
|
||||
}
|
||||
|
||||
result, err := h.storageService.GetUploadURL(
|
||||
c.UserContext(),
|
||||
req.Purpose,
|
||||
req.FileName,
|
||||
req.ContentType,
|
||||
)
|
||||
if err != nil {
|
||||
return errors.New(errors.CodeStorageUploadFailed, err.Error())
|
||||
}
|
||||
|
||||
return response.Success(c, result)
|
||||
}
|
||||
```
|
||||
|
||||
## 在 Worker 中使用
|
||||
|
||||
```go
|
||||
func (h *TaskHandler) HandleTask(ctx context.Context, task *asynq.Task) error {
|
||||
// 从任务记录获取文件路径
|
||||
fileKey := importTask.StorageKey
|
||||
|
||||
// 下载到临时文件
|
||||
localPath, cleanup, err := h.storageService.DownloadToTemp(ctx, fileKey)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer cleanup()
|
||||
|
||||
// 解析文件
|
||||
f, _ := os.Open(localPath)
|
||||
defer f.Close()
|
||||
|
||||
// 处理文件内容...
|
||||
}
|
||||
```
|
||||
|
||||
## 测试验证
|
||||
|
||||
运行对象存储功能测试:
|
||||
|
||||
```bash
|
||||
go run scripts/test_storage.go
|
||||
```
|
||||
|
||||
测试内容包括:
|
||||
1. 生成预签名上传 URL
|
||||
2. 上传测试文件
|
||||
3. 检查文件是否存在
|
||||
4. 下载到临时文件
|
||||
5. 删除测试文件
|
||||
250
docs/object-storage/前端接入指南.md
Normal file
250
docs/object-storage/前端接入指南.md
Normal file
@@ -0,0 +1,250 @@
|
||||
# 对象存储前端接入指南
|
||||
|
||||
## 文件上传流程
|
||||
|
||||
```
|
||||
前端 后端 API 对象存储
|
||||
│ │ │
|
||||
│ 1. POST /storage/upload-url │
|
||||
│ {file_name, content_type, purpose} │
|
||||
│ ─────────────────────────► │
|
||||
│ │ │
|
||||
│ 2. 返回 {upload_url, file_key, expires_in} │
|
||||
│ ◄───────────────────────── │
|
||||
│ │ │
|
||||
│ 3. PUT upload_url (文件内容) │
|
||||
│ ─────────────────────────────────────────────────► │
|
||||
│ │ │
|
||||
│ 4. 上传成功 (200 OK) │
|
||||
│ ◄───────────────────────────────────────────────── │
|
||||
│ │ │
|
||||
│ 5. POST /iot-cards/import │
|
||||
│ {carrier_id, batch_no, file_key} │
|
||||
│ ─────────────────────────► │
|
||||
│ │ │
|
||||
│ 6. 返回任务创建成功 │
|
||||
│ ◄───────────────────────── │
|
||||
```
|
||||
|
||||
## 获取预签名 URL 接口
|
||||
|
||||
### 请求
|
||||
|
||||
```http
|
||||
POST /api/admin/storage/upload-url
|
||||
Content-Type: application/json
|
||||
Authorization: Bearer {token}
|
||||
|
||||
{
|
||||
"file_name": "cards.csv",
|
||||
"content_type": "text/csv",
|
||||
"purpose": "iot_import"
|
||||
}
|
||||
```
|
||||
|
||||
### 响应
|
||||
|
||||
```json
|
||||
{
|
||||
"code": 0,
|
||||
"message": "成功",
|
||||
"data": {
|
||||
"upload_url": "http://obs-helf.cucloud.cn/cmp/imports/2025/01/24/abc123.csv?X-Amz-Algorithm=...",
|
||||
"file_key": "imports/2025/01/24/abc123.csv",
|
||||
"expires_in": 900
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### purpose 可选值
|
||||
|
||||
| 值 | 说明 | 生成路径 |
|
||||
|---|------|---------|
|
||||
| iot_import | ICCID 导入 | imports/YYYY/MM/DD/uuid.csv |
|
||||
| export | 数据导出 | exports/YYYY/MM/DD/uuid.xlsx |
|
||||
| attachment | 附件上传 | attachments/YYYY/MM/DD/uuid.ext |
|
||||
|
||||
## 使用预签名 URL 上传文件
|
||||
|
||||
获取到 `upload_url` 后,直接使用 PUT 请求上传文件到对象存储:
|
||||
|
||||
```javascript
|
||||
const response = await fetch(upload_url, {
|
||||
method: 'PUT',
|
||||
headers: {
|
||||
'Content-Type': content_type
|
||||
},
|
||||
body: file
|
||||
});
|
||||
|
||||
if (response.ok) {
|
||||
console.log('上传成功');
|
||||
} else {
|
||||
console.error('上传失败:', response.status);
|
||||
}
|
||||
```
|
||||
|
||||
## ICCID 导入接口变更(BREAKING CHANGE)
|
||||
|
||||
### 变更前
|
||||
|
||||
```http
|
||||
POST /api/admin/iot-cards/import
|
||||
Content-Type: multipart/form-data
|
||||
|
||||
carrier_id=1
|
||||
batch_no=BATCH-2025-01
|
||||
file=@cards.csv
|
||||
```
|
||||
|
||||
### 变更后
|
||||
|
||||
```http
|
||||
POST /api/admin/iot-cards/import
|
||||
Content-Type: application/json
|
||||
Authorization: Bearer {token}
|
||||
|
||||
{
|
||||
"carrier_id": 1,
|
||||
"batch_no": "BATCH-2025-01",
|
||||
"file_key": "imports/2025/01/24/abc123.csv"
|
||||
}
|
||||
```
|
||||
|
||||
## 完整代码示例(TypeScript)
|
||||
|
||||
```typescript
|
||||
interface UploadURLResponse {
|
||||
upload_url: string;
|
||||
file_key: string;
|
||||
expires_in: number;
|
||||
}
|
||||
|
||||
async function uploadAndImportCards(
|
||||
file: File,
|
||||
carrierId: number,
|
||||
batchNo: string
|
||||
): Promise<void> {
|
||||
// 1. 获取预签名上传 URL
|
||||
const urlResponse = await fetch('/api/admin/storage/upload-url', {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
'Authorization': `Bearer ${getToken()}`
|
||||
},
|
||||
body: JSON.stringify({
|
||||
file_name: file.name,
|
||||
content_type: file.type || 'text/csv',
|
||||
purpose: 'iot_import'
|
||||
})
|
||||
});
|
||||
|
||||
if (!urlResponse.ok) {
|
||||
throw new Error('获取上传 URL 失败');
|
||||
}
|
||||
|
||||
const { data } = await urlResponse.json();
|
||||
const { upload_url, file_key } = data as UploadURLResponse;
|
||||
|
||||
// 2. 上传文件到对象存储
|
||||
const uploadResponse = await fetch(upload_url, {
|
||||
method: 'PUT',
|
||||
headers: {
|
||||
'Content-Type': file.type || 'text/csv'
|
||||
},
|
||||
body: file
|
||||
});
|
||||
|
||||
if (!uploadResponse.ok) {
|
||||
throw new Error('文件上传失败');
|
||||
}
|
||||
|
||||
// 3. 调用导入接口
|
||||
const importResponse = await fetch('/api/admin/iot-cards/import', {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
'Authorization': `Bearer ${getToken()}`
|
||||
},
|
||||
body: JSON.stringify({
|
||||
carrier_id: carrierId,
|
||||
batch_no: batchNo,
|
||||
file_key: file_key
|
||||
})
|
||||
});
|
||||
|
||||
if (!importResponse.ok) {
|
||||
throw new Error('导入任务创建失败');
|
||||
}
|
||||
|
||||
console.log('导入任务已创建');
|
||||
}
|
||||
```
|
||||
|
||||
## 错误处理和重试策略
|
||||
|
||||
### 预签名 URL 过期
|
||||
|
||||
预签名 URL 有效期为 15 分钟。如果上传时 URL 已过期,需要重新获取:
|
||||
|
||||
```typescript
|
||||
async function uploadWithRetry(file: File, purpose: string, maxRetries = 3) {
|
||||
for (let i = 0; i < maxRetries; i++) {
|
||||
const { upload_url, file_key } = await getUploadURL(file.name, file.type, purpose);
|
||||
|
||||
try {
|
||||
await uploadFile(upload_url, file);
|
||||
return file_key;
|
||||
} catch (error) {
|
||||
if (i === maxRetries - 1) throw error;
|
||||
console.warn(`上传失败,重试 ${i + 1}/${maxRetries}`);
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 网络错误
|
||||
|
||||
对象存储上传可能因网络问题失败,建议实现重试机制:
|
||||
|
||||
```typescript
|
||||
async function uploadFile(url: string, file: File, retries = 3) {
|
||||
for (let i = 0; i < retries; i++) {
|
||||
try {
|
||||
const response = await fetch(url, {
|
||||
method: 'PUT',
|
||||
headers: { 'Content-Type': file.type },
|
||||
body: file
|
||||
});
|
||||
|
||||
if (response.ok) return;
|
||||
|
||||
if (response.status >= 500) {
|
||||
// 服务端错误,可重试
|
||||
continue;
|
||||
}
|
||||
|
||||
throw new Error(`上传失败: ${response.status}`);
|
||||
} catch (error) {
|
||||
if (i === retries - 1) throw error;
|
||||
await new Promise(r => setTimeout(r, 1000 * (i + 1)));
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## 常见问题
|
||||
|
||||
### Q: 上传时报 CORS 错误
|
||||
|
||||
确保对象存储已配置 CORS 规则允许前端域名访问。
|
||||
|
||||
### Q: 预签名 URL 无法使用
|
||||
|
||||
1. 检查 URL 是否过期(15 分钟有效期)
|
||||
2. 确保 Content-Type 与获取 URL 时指定的一致
|
||||
3. 检查文件大小是否超过限制
|
||||
|
||||
### Q: file_key 可以重复使用吗
|
||||
|
||||
可以。file_key 一旦上传成功就永久有效,可以在多个业务接口中使用同一个 file_key。
|
||||
6
go.mod
6
go.mod
@@ -3,6 +3,7 @@ module github.com/break/junhong_cmp_fiber
|
||||
go 1.25
|
||||
|
||||
require (
|
||||
github.com/aws/aws-sdk-go v1.55.5
|
||||
github.com/bytedance/sonic v1.14.2
|
||||
github.com/fsnotify/fsnotify v1.9.0
|
||||
github.com/go-playground/validator/v10 v10.28.0
|
||||
@@ -12,7 +13,6 @@ require (
|
||||
github.com/golang-migrate/migrate/v4 v4.19.0
|
||||
github.com/google/uuid v1.6.0
|
||||
github.com/hibiken/asynq v0.25.1
|
||||
github.com/lib/pq v1.10.9
|
||||
github.com/redis/go-redis/v9 v9.16.0
|
||||
github.com/spf13/viper v1.21.0
|
||||
github.com/stretchr/testify v1.11.1
|
||||
@@ -25,6 +25,7 @@ require (
|
||||
golang.org/x/crypto v0.44.0
|
||||
gopkg.in/natefinch/lumberjack.v2 v2.2.1
|
||||
gopkg.in/yaml.v3 v3.0.1
|
||||
gorm.io/datatypes v1.2.7
|
||||
gorm.io/driver/postgres v1.6.0
|
||||
gorm.io/driver/sqlite v1.6.0
|
||||
gorm.io/gorm v1.31.1
|
||||
@@ -71,9 +72,11 @@ require (
|
||||
github.com/jackc/puddle/v2 v2.2.2 // indirect
|
||||
github.com/jinzhu/inflection v1.0.0 // indirect
|
||||
github.com/jinzhu/now v1.1.5 // indirect
|
||||
github.com/jmespath/go-jmespath v0.4.0 // indirect
|
||||
github.com/klauspost/compress v1.18.0 // indirect
|
||||
github.com/klauspost/cpuid/v2 v2.2.9 // indirect
|
||||
github.com/leodido/go-urn v1.4.0 // indirect
|
||||
github.com/lib/pq v1.10.9 // indirect
|
||||
github.com/lufia/plan9stats v0.0.0-20211012122336-39d0f177ccd0 // indirect
|
||||
github.com/magiconair/properties v1.8.10 // indirect
|
||||
github.com/mattn/go-colorable v0.1.13 // indirect
|
||||
@@ -131,6 +134,5 @@ require (
|
||||
google.golang.org/grpc v1.75.1 // indirect
|
||||
google.golang.org/protobuf v1.36.10 // indirect
|
||||
gopkg.in/yaml.v2 v2.4.0 // indirect
|
||||
gorm.io/datatypes v1.2.7 // indirect
|
||||
gorm.io/driver/mysql v1.5.6 // indirect
|
||||
)
|
||||
|
||||
15
go.sum
15
go.sum
@@ -10,6 +10,8 @@ github.com/Microsoft/go-winio v0.6.2 h1:F2VQgta7ecxGYO8k3ZZz3RS8fVIXVxONVUPlNERo
|
||||
github.com/Microsoft/go-winio v0.6.2/go.mod h1:yd8OoFMLzJbo9gZq8j5qaps8bJ9aShtEA8Ipt1oGCvU=
|
||||
github.com/andybalholm/brotli v1.2.0 h1:ukwgCxwYrmACq68yiUqwIWnGY0cTPox/M94sVwToPjQ=
|
||||
github.com/andybalholm/brotli v1.2.0/go.mod h1:rzTDkvFWvIrjDXZHkuS16NPggd91W3kUSvPlQ1pLaKY=
|
||||
github.com/aws/aws-sdk-go v1.55.5 h1:KKUZBfBoyqy5d3swXyiC7Q76ic40rYcbqH7qjh59kzU=
|
||||
github.com/aws/aws-sdk-go v1.55.5/go.mod h1:eRwEWoyTWFMVYVQzKMNHWP5/RV4xIUGMQfXQHfHkpNU=
|
||||
github.com/bool64/dev v0.2.39 h1:kP8DnMGlWXhGYJEZE/J0l/gVBdbuhoPGL+MJG4QbofE=
|
||||
github.com/bool64/dev v0.2.39/go.mod h1:iJbh1y/HkunEPhgebWRNcs8wfGq7sjvJ6W5iabL8ACg=
|
||||
github.com/bool64/shared v0.1.5 h1:fp3eUhBsrSjNCQPcSdQqZxxh9bBwrYiZ+zOKFkM0/2E=
|
||||
@@ -97,6 +99,10 @@ github.com/golang-jwt/jwt/v5 v5.3.0 h1:pv4AsKCKKZuqlgs5sUmn4x8UlGa0kEVt/puTpKx9v
|
||||
github.com/golang-jwt/jwt/v5 v5.3.0/go.mod h1:fxCRLWMO43lRc8nhHWY6LGqRcf+1gQWArsqaEUEa5bE=
|
||||
github.com/golang-migrate/migrate/v4 v4.19.0 h1:RcjOnCGz3Or6HQYEJ/EEVLfWnmw9KnoigPSjzhCuaSE=
|
||||
github.com/golang-migrate/migrate/v4 v4.19.0/go.mod h1:9dyEcu+hO+G9hPSw8AIg50yg622pXJsoHItQnDGZkI0=
|
||||
github.com/golang-sql/civil v0.0.0-20220223132316-b832511892a9 h1:au07oEsX2xN0ktxqI+Sida1w446QrXBRJ0nee3SNZlA=
|
||||
github.com/golang-sql/civil v0.0.0-20220223132316-b832511892a9/go.mod h1:8vg3r2VgvsThLBIFL93Qb5yWzgyZWhEmBwUJWevAkK0=
|
||||
github.com/golang-sql/sqlexp v0.1.0 h1:ZCD6MBpcuOVfGVqsEmY5/4FtYiKz6tSyUv9LPEDei6A=
|
||||
github.com/golang-sql/sqlexp v0.1.0/go.mod h1:J4ad9Vo8ZCWQ2GMrC4UCQy1JpCbwU9m3EOqtpKwwwHI=
|
||||
github.com/google/go-cmp v0.5.6/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
|
||||
github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
|
||||
github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU=
|
||||
@@ -125,6 +131,10 @@ github.com/jinzhu/inflection v1.0.0 h1:K317FqzuhWc8YvSVlFMCCUb36O/S9MCKRDI7QkRKD
|
||||
github.com/jinzhu/inflection v1.0.0/go.mod h1:h+uFLlag+Qp1Va5pdKtLDYj+kHp5pxUVkryuEj+Srlc=
|
||||
github.com/jinzhu/now v1.1.5 h1:/o9tlHleP7gOFmsnYNz3RGnqzefHA47wQpKrrdTIwXQ=
|
||||
github.com/jinzhu/now v1.1.5/go.mod h1:d3SSVoowX0Lcu0IBviAWJpolVfI5UJVZZ7cO71lE/z8=
|
||||
github.com/jmespath/go-jmespath v0.4.0 h1:BEgLn5cpjn8UN1mAw4NjwDrS35OdebyEtFe+9YPoQUg=
|
||||
github.com/jmespath/go-jmespath v0.4.0/go.mod h1:T8mJZnbsbmF+m6zOOFylbeCJqk5+pHWvzYPziyZiYoo=
|
||||
github.com/jmespath/go-jmespath/internal/testify v1.5.1 h1:shLQSRRSCCPj3f2gpwzGwWFoC7ycTf1rcQZHOlsJ6N8=
|
||||
github.com/jmespath/go-jmespath/internal/testify v1.5.1/go.mod h1:L3OGu8Wl2/fWfCI6z80xFu9LTZmf1ZRjMHUOPmWr69U=
|
||||
github.com/klauspost/compress v1.18.0 h1:c/Cqfb0r+Yi+JtIEq73FWXVkRonBlf0CRNYc8Zttxdo=
|
||||
github.com/klauspost/compress v1.18.0/go.mod h1:2Pp+KzxcywXVXMr50+X0Q/Lsb43OQHYWRCY2AiWywWQ=
|
||||
github.com/klauspost/cpuid/v2 v2.2.9 h1:66ze0taIn2H33fBvCkXuv9BmCwDfafmiIVpKV9kKGuY=
|
||||
@@ -152,6 +162,8 @@ github.com/mattn/go-sqlite3 v1.14.22 h1:2gZY6PC6kBnID23Tichd1K+Z0oS6nE/XwU+Vz/5o
|
||||
github.com/mattn/go-sqlite3 v1.14.22/go.mod h1:Uh1q+B4BYcTPb+yiD3kU8Ct7aC0hY9fxUwlHK0RXw+Y=
|
||||
github.com/mdelapenya/tlscert v0.2.0 h1:7H81W6Z/4weDvZBNOfQte5GpIMo0lGYEeWbkGp5LJHI=
|
||||
github.com/mdelapenya/tlscert v0.2.0/go.mod h1:O4njj3ELLnJjGdkN7M/vIVCpZ+Cf0L6muqOG4tLSl8o=
|
||||
github.com/microsoft/go-mssqldb v1.7.2 h1:CHkFJiObW7ItKTJfHo1QX7QBBD1iV+mn1eOyRP3b/PA=
|
||||
github.com/microsoft/go-mssqldb v1.7.2/go.mod h1:kOvZKUdrhhFQmxLZqbwUV0rHkNkZpthMITIb2Ko1IoA=
|
||||
github.com/moby/docker-image-spec v1.3.1 h1:jMKff3w6PgbfSa69GfNg+zN/XLhfXJGnEx3Nl2EsFP0=
|
||||
github.com/moby/docker-image-spec v1.3.1/go.mod h1:eKmb5VW8vQEh/BAr2yvVNvuiJuY6UIocYsFu/DxxRpo=
|
||||
github.com/moby/go-archive v0.1.0 h1:Kk/5rdW/g+H8NHdJW2gsXyZ7UnzvJNOy6VKJqueWdcQ=
|
||||
@@ -324,6 +336,7 @@ gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntN
|
||||
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q=
|
||||
gopkg.in/natefinch/lumberjack.v2 v2.2.1 h1:bBRl1b0OH9s/DuPhuXpNl+VtCaJXFZ5/uEFST95x9zc=
|
||||
gopkg.in/natefinch/lumberjack.v2 v2.2.1/go.mod h1:YD8tP3GAjkrDg1eZH7EGmyESg/lsYskCTPBJVb9jqSc=
|
||||
gopkg.in/yaml.v2 v2.2.8/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
|
||||
gopkg.in/yaml.v2 v2.4.0 h1:D8xgwECY7CYvx+Y2n4sBz93Jn9JRvxdiyyo8CTfuKaY=
|
||||
gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ=
|
||||
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
|
||||
@@ -337,6 +350,8 @@ gorm.io/driver/postgres v1.6.0 h1:2dxzU8xJ+ivvqTRph34QX+WrRaJlmfyPqXmoGVjMBa4=
|
||||
gorm.io/driver/postgres v1.6.0/go.mod h1:vUw0mrGgrTK+uPHEhAdV4sfFELrByKVGnaVRkXDhtWo=
|
||||
gorm.io/driver/sqlite v1.6.0 h1:WHRRrIiulaPiPFmDcod6prc4l2VGVWHz80KspNsxSfQ=
|
||||
gorm.io/driver/sqlite v1.6.0/go.mod h1:AO9V1qIQddBESngQUKWL9yoH93HIeA1X6V633rBwyT8=
|
||||
gorm.io/driver/sqlserver v1.6.0 h1:VZOBQVsVhkHU/NzNhRJKoANt5pZGQAS1Bwc6m6dgfnc=
|
||||
gorm.io/driver/sqlserver v1.6.0/go.mod h1:WQzt4IJo/WHKnckU9jXBLMJIVNMVeTu25dnOzehntWw=
|
||||
gorm.io/gorm v1.25.7/go.mod h1:hbnx/Oo0ChWMn1BIhpy1oYozzpM15i4YPuHDmfYtwg8=
|
||||
gorm.io/gorm v1.31.1 h1:7CA8FTFz/gRfgqgpeKIBcervUn3xSyPUmr6B2WXJ7kg=
|
||||
gorm.io/gorm v1.31.1/go.mod h1:XyQVbO2k6YkOis7C2437jSit3SsDK72s7n7rsSHd+Gs=
|
||||
|
||||
@@ -4,6 +4,7 @@ import (
|
||||
"github.com/break/junhong_cmp_fiber/internal/service/verification"
|
||||
"github.com/break/junhong_cmp_fiber/pkg/auth"
|
||||
"github.com/break/junhong_cmp_fiber/pkg/queue"
|
||||
"github.com/break/junhong_cmp_fiber/pkg/storage"
|
||||
"github.com/redis/go-redis/v9"
|
||||
"go.uber.org/zap"
|
||||
"gorm.io/gorm"
|
||||
@@ -19,4 +20,5 @@ type Dependencies struct {
|
||||
TokenManager *auth.TokenManager // Token 管理器(后台和H5认证)
|
||||
VerificationService *verification.Service // 验证码服务
|
||||
QueueClient *queue.Client // Asynq 任务队列客户端
|
||||
StorageService *storage.Service // 对象存储服务(可选,配置缺失时为 nil)
|
||||
}
|
||||
|
||||
@@ -29,5 +29,6 @@ func initHandlers(svc *services, deps *Dependencies) *Handlers {
|
||||
IotCard: admin.NewIotCardHandler(svc.IotCard),
|
||||
IotCardImport: admin.NewIotCardImportHandler(svc.IotCardImport),
|
||||
AssetAllocationRecord: admin.NewAssetAllocationRecordHandler(svc.AssetAllocationRecord),
|
||||
Storage: admin.NewStorageHandler(deps.StorageService),
|
||||
}
|
||||
}
|
||||
|
||||
@@ -27,6 +27,7 @@ type Handlers struct {
|
||||
IotCard *admin.IotCardHandler
|
||||
IotCardImport *admin.IotCardImportHandler
|
||||
AssetAllocationRecord *admin.AssetAllocationRecordHandler
|
||||
Storage *admin.StorageHandler
|
||||
}
|
||||
|
||||
// Middlewares 封装所有中间件
|
||||
|
||||
@@ -16,7 +16,9 @@ type IotCardImportHandler struct {
|
||||
}
|
||||
|
||||
func NewIotCardImportHandler(service *iotCardImportService.Service) *IotCardImportHandler {
|
||||
return &IotCardImportHandler{service: service}
|
||||
return &IotCardImportHandler{
|
||||
service: service,
|
||||
}
|
||||
}
|
||||
|
||||
func (h *IotCardImportHandler) Import(c *fiber.Ctx) error {
|
||||
@@ -25,18 +27,11 @@ func (h *IotCardImportHandler) Import(c *fiber.Ctx) error {
|
||||
return errors.New(errors.CodeInvalidParam, "请求参数解析失败")
|
||||
}
|
||||
|
||||
file, err := c.FormFile("file")
|
||||
if err != nil {
|
||||
return errors.New(errors.CodeInvalidParam, "请上传 CSV 文件")
|
||||
if req.FileKey == "" {
|
||||
return errors.New(errors.CodeInvalidParam, "文件路径不能为空")
|
||||
}
|
||||
|
||||
f, err := file.Open()
|
||||
if err != nil {
|
||||
return errors.New(errors.CodeInvalidParam, "无法读取上传文件")
|
||||
}
|
||||
defer f.Close()
|
||||
|
||||
result, err := h.service.CreateImportTask(c.UserContext(), &req, f, file.Filename)
|
||||
result, err := h.service.CreateImportTask(c.UserContext(), &req)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
40
internal/handler/admin/storage.go
Normal file
40
internal/handler/admin/storage.go
Normal file
@@ -0,0 +1,40 @@
|
||||
package admin
|
||||
|
||||
import (
|
||||
"github.com/gofiber/fiber/v2"
|
||||
|
||||
"github.com/break/junhong_cmp_fiber/internal/model/dto"
|
||||
"github.com/break/junhong_cmp_fiber/pkg/errors"
|
||||
"github.com/break/junhong_cmp_fiber/pkg/response"
|
||||
"github.com/break/junhong_cmp_fiber/pkg/storage"
|
||||
)
|
||||
|
||||
type StorageHandler struct {
|
||||
service *storage.Service
|
||||
}
|
||||
|
||||
func NewStorageHandler(service *storage.Service) *StorageHandler {
|
||||
return &StorageHandler{service: service}
|
||||
}
|
||||
|
||||
func (h *StorageHandler) GetUploadURL(c *fiber.Ctx) error {
|
||||
if h.service == nil {
|
||||
return errors.New(errors.CodeInternalError, "对象存储服务未配置")
|
||||
}
|
||||
|
||||
var req dto.GetUploadURLRequest
|
||||
if err := c.BodyParser(&req); err != nil {
|
||||
return errors.New(errors.CodeInvalidParam, "请求参数解析失败")
|
||||
}
|
||||
|
||||
result, err := h.service.GetUploadURL(c.UserContext(), req.Purpose, req.FileName, req.ContentType)
|
||||
if err != nil {
|
||||
return errors.New(errors.CodeInternalError, err.Error())
|
||||
}
|
||||
|
||||
return response.Success(c, dto.GetUploadURLResponse{
|
||||
UploadURL: result.URL,
|
||||
FileKey: result.FileKey,
|
||||
ExpiresIn: result.ExpiresIn,
|
||||
})
|
||||
}
|
||||
@@ -52,8 +52,9 @@ type ListStandaloneIotCardResponse struct {
|
||||
}
|
||||
|
||||
type ImportIotCardRequest struct {
|
||||
CarrierID uint `json:"carrier_id" form:"carrier_id" validate:"required,min=1" required:"true" minimum:"1" description:"运营商ID"`
|
||||
BatchNo string `json:"batch_no" form:"batch_no" validate:"omitempty,max=100" maxLength:"100" description:"批次号"`
|
||||
CarrierID uint `json:"carrier_id" validate:"required,min=1" required:"true" minimum:"1" description:"运营商ID"`
|
||||
BatchNo string `json:"batch_no" validate:"omitempty,max=100" maxLength:"100" description:"批次号"`
|
||||
FileKey string `json:"file_key" validate:"required,min=1,max=500" required:"true" minLength:"1" maxLength:"500" description:"对象存储文件路径(通过 /storage/upload-url 获取)"`
|
||||
}
|
||||
|
||||
type ImportIotCardResponse struct {
|
||||
@@ -102,6 +103,7 @@ type ListImportTaskResponse struct {
|
||||
type ImportResultItemDTO struct {
|
||||
Line int `json:"line" description:"行号"`
|
||||
ICCID string `json:"iccid" description:"ICCID"`
|
||||
MSISDN string `json:"msisdn,omitempty" description:"接入号"`
|
||||
Reason string `json:"reason" description:"原因"`
|
||||
}
|
||||
|
||||
|
||||
13
internal/model/dto/storage_dto.go
Normal file
13
internal/model/dto/storage_dto.go
Normal file
@@ -0,0 +1,13 @@
|
||||
package dto
|
||||
|
||||
type GetUploadURLRequest struct {
|
||||
FileName string `json:"file_name" validate:"required,min=1,max=255" required:"true" minLength:"1" maxLength:"255" description:"文件名(如:cards.csv)"`
|
||||
ContentType string `json:"content_type" validate:"omitempty,max=100" maxLength:"100" description:"文件 MIME 类型(如:text/csv),留空则自动推断"`
|
||||
Purpose string `json:"purpose" validate:"required,oneof=iot_import export attachment" required:"true" description:"文件用途 (iot_import:ICCID导入, export:数据导出, attachment:附件)"`
|
||||
}
|
||||
|
||||
type GetUploadURLResponse struct {
|
||||
UploadURL string `json:"upload_url" description:"预签名上传 URL,使用 PUT 方法上传文件"`
|
||||
FileKey string `json:"file_key" description:"文件路径标识,上传成功后用于调用业务接口"`
|
||||
ExpiresIn int `json:"expires_in" description:"URL 有效期(秒)"`
|
||||
}
|
||||
@@ -10,38 +10,46 @@ import (
|
||||
|
||||
type IotCardImportTask struct {
|
||||
gorm.Model
|
||||
BaseModel `gorm:"embedded"`
|
||||
TaskNo string `gorm:"column:task_no;type:varchar(50);uniqueIndex:idx_import_task_no,where:deleted_at IS NULL;not null;comment:任务编号(IMP-YYYYMMDD-XXXXXX)" json:"task_no"`
|
||||
Status int `gorm:"column:status;type:int;default:1;not null;comment:任务状态 1-待处理 2-处理中 3-已完成 4-失败" json:"status"`
|
||||
CarrierID uint `gorm:"column:carrier_id;index;not null;comment:运营商ID" json:"carrier_id"`
|
||||
CarrierType string `gorm:"column:carrier_type;type:varchar(20);not null;comment:运营商类型(CMCC/CUCC/CTCC/CBN)" json:"carrier_type"`
|
||||
BatchNo string `gorm:"column:batch_no;type:varchar(100);comment:批次号" json:"batch_no"`
|
||||
FileName string `gorm:"column:file_name;type:varchar(255);comment:原始文件名" json:"file_name"`
|
||||
TotalCount int `gorm:"column:total_count;type:int;default:0;not null;comment:总数" json:"total_count"`
|
||||
SuccessCount int `gorm:"column:success_count;type:int;default:0;not null;comment:成功数" json:"success_count"`
|
||||
SkipCount int `gorm:"column:skip_count;type:int;default:0;not null;comment:跳过数" json:"skip_count"`
|
||||
FailCount int `gorm:"column:fail_count;type:int;default:0;not null;comment:失败数" json:"fail_count"`
|
||||
SkippedItems ImportResultItems `gorm:"column:skipped_items;type:jsonb;comment:跳过记录详情" json:"skipped_items"`
|
||||
FailedItems ImportResultItems `gorm:"column:failed_items;type:jsonb;comment:失败记录详情" json:"failed_items"`
|
||||
StartedAt *time.Time `gorm:"column:started_at;comment:开始处理时间" json:"started_at"`
|
||||
CompletedAt *time.Time `gorm:"column:completed_at;comment:完成时间" json:"completed_at"`
|
||||
ErrorMessage string `gorm:"column:error_message;type:text;comment:任务级错误信息" json:"error_message"`
|
||||
ShopID *uint `gorm:"column:shop_id;index;comment:店铺ID(发起导入的店铺)" json:"shop_id,omitempty"`
|
||||
ICCIDList ICCIDListJSON `gorm:"column:iccid_list;type:jsonb;comment:待导入ICCID列表" json:"-"`
|
||||
BaseModel `gorm:"embedded"`
|
||||
TaskNo string `gorm:"column:task_no;type:varchar(50);uniqueIndex:idx_import_task_no,where:deleted_at IS NULL;not null;comment:任务编号(IMP-YYYYMMDD-XXXXXX)" json:"task_no"`
|
||||
Status int `gorm:"column:status;type:int;default:1;not null;comment:任务状态 1-待处理 2-处理中 3-已完成 4-失败" json:"status"`
|
||||
CarrierID uint `gorm:"column:carrier_id;index;not null;comment:运营商ID" json:"carrier_id"`
|
||||
CarrierType string `gorm:"column:carrier_type;type:varchar(20);not null;comment:运营商类型(CMCC/CUCC/CTCC/CBN)" json:"carrier_type"`
|
||||
BatchNo string `gorm:"column:batch_no;type:varchar(100);comment:批次号" json:"batch_no"`
|
||||
FileName string `gorm:"column:file_name;type:varchar(255);comment:原始文件名" json:"file_name"`
|
||||
TotalCount int `gorm:"column:total_count;type:int;default:0;not null;comment:总数" json:"total_count"`
|
||||
SuccessCount int `gorm:"column:success_count;type:int;default:0;not null;comment:成功数" json:"success_count"`
|
||||
SkipCount int `gorm:"column:skip_count;type:int;default:0;not null;comment:跳过数" json:"skip_count"`
|
||||
FailCount int `gorm:"column:fail_count;type:int;default:0;not null;comment:失败数" json:"fail_count"`
|
||||
SkippedItems ImportResultItems `gorm:"column:skipped_items;type:jsonb;comment:跳过记录详情" json:"skipped_items"`
|
||||
FailedItems ImportResultItems `gorm:"column:failed_items;type:jsonb;comment:失败记录详情" json:"failed_items"`
|
||||
StartedAt *time.Time `gorm:"column:started_at;comment:开始处理时间" json:"started_at"`
|
||||
CompletedAt *time.Time `gorm:"column:completed_at;comment:完成时间" json:"completed_at"`
|
||||
ErrorMessage string `gorm:"column:error_message;type:text;comment:任务级错误信息" json:"error_message"`
|
||||
ShopID *uint `gorm:"column:shop_id;index;comment:店铺ID(发起导入的店铺)" json:"shop_id,omitempty"`
|
||||
CardList CardListJSON `gorm:"column:card_list;type:jsonb;comment:待导入卡列表[{iccid,msisdn}]" json:"-"`
|
||||
StorageBucket string `gorm:"column:storage_bucket;type:varchar(100);comment:对象存储桶名" json:"storage_bucket,omitempty"`
|
||||
StorageKey string `gorm:"column:storage_key;type:varchar(500);comment:对象存储文件路径" json:"storage_key,omitempty"`
|
||||
}
|
||||
|
||||
type ICCIDListJSON []string
|
||||
// CardItem 卡信息(ICCID + MSISDN)
|
||||
type CardItem struct {
|
||||
ICCID string `json:"iccid"`
|
||||
MSISDN string `json:"msisdn"`
|
||||
}
|
||||
|
||||
func (list ICCIDListJSON) Value() (driver.Value, error) {
|
||||
type CardListJSON []CardItem
|
||||
|
||||
func (list CardListJSON) Value() (driver.Value, error) {
|
||||
if list == nil {
|
||||
return "[]", nil
|
||||
}
|
||||
return json.Marshal(list)
|
||||
}
|
||||
|
||||
func (list *ICCIDListJSON) Scan(value any) error {
|
||||
func (list *CardListJSON) Scan(value any) error {
|
||||
if value == nil {
|
||||
*list = ICCIDListJSON{}
|
||||
*list = CardListJSON{}
|
||||
return nil
|
||||
}
|
||||
bytes, ok := value.([]byte)
|
||||
@@ -58,6 +66,7 @@ func (IotCardImportTask) TableName() string {
|
||||
type ImportResultItem struct {
|
||||
Line int `json:"line"`
|
||||
ICCID string `json:"iccid"`
|
||||
MSISDN string `json:"msisdn,omitempty"`
|
||||
Reason string `json:"reason"`
|
||||
}
|
||||
|
||||
|
||||
@@ -58,6 +58,9 @@ func RegisterAdminRoutes(router fiber.Router, handlers *bootstrap.Handlers, midd
|
||||
if handlers.AssetAllocationRecord != nil {
|
||||
registerAssetAllocationRecordRoutes(authGroup, handlers.AssetAllocationRecord, doc, basePath)
|
||||
}
|
||||
if handlers.Storage != nil {
|
||||
registerStorageRoutes(authGroup, handlers.Storage, doc, basePath)
|
||||
}
|
||||
}
|
||||
|
||||
func registerAdminAuthRoutes(router fiber.Router, handler interface{}, authMiddleware fiber.Handler, doc *openapi.Generator, basePath string) {
|
||||
|
||||
@@ -21,11 +21,36 @@ func registerIotCardRoutes(router fiber.Router, handler *admin.IotCardHandler, i
|
||||
})
|
||||
|
||||
Register(iotCards, doc, groupPath, "POST", "/import", importHandler.Import, RouteSpec{
|
||||
Summary: "批量导入ICCID",
|
||||
Tags: []string{"IoT卡管理"},
|
||||
Input: new(dto.ImportIotCardRequest),
|
||||
Output: new(dto.ImportIotCardResponse),
|
||||
Auth: true,
|
||||
Summary: "批量导入IoT卡(ICCID+MSISDN)",
|
||||
Description: `## ⚠️ 接口变更说明(BREAKING CHANGE)
|
||||
|
||||
本接口已从 ` + "`multipart/form-data`" + ` 改为 ` + "`application/json`" + `。
|
||||
|
||||
### 完整导入流程
|
||||
|
||||
1. **获取上传 URL**: 调用 ` + "`POST /api/admin/storage/upload-url`" + `
|
||||
2. **上传 CSV 文件**: 使用预签名 URL 上传文件到对象存储
|
||||
3. **调用本接口**: 使用返回的 ` + "`file_key`" + ` 提交导入任务
|
||||
|
||||
### 请求示例
|
||||
|
||||
` + "```" + `json
|
||||
{
|
||||
"carrier_id": 1,
|
||||
"batch_no": "BATCH-2025-01",
|
||||
"file_key": "imports/2025/01/24/abc123.csv"
|
||||
}
|
||||
` + "```" + `
|
||||
|
||||
### CSV 文件格式
|
||||
|
||||
- 必须包含两列:` + "`iccid`" + `, ` + "`msisdn`" + `
|
||||
- 首行为表头
|
||||
- 编码:UTF-8`,
|
||||
Tags: []string{"IoT卡管理"},
|
||||
Input: new(dto.ImportIotCardRequest),
|
||||
Output: new(dto.ImportIotCardResponse),
|
||||
Auth: true,
|
||||
})
|
||||
|
||||
Register(iotCards, doc, groupPath, "GET", "/import-tasks", importHandler.List, RouteSpec{
|
||||
|
||||
@@ -8,13 +8,22 @@ import (
|
||||
"github.com/break/junhong_cmp_fiber/pkg/openapi"
|
||||
)
|
||||
|
||||
// FileUploadField 定义文件上传字段
|
||||
type FileUploadField struct {
|
||||
Name string // 字段名
|
||||
Description string // 字段描述
|
||||
Required bool // 是否必填
|
||||
}
|
||||
|
||||
// RouteSpec 定义接口文档元数据
|
||||
type RouteSpec struct {
|
||||
Summary string
|
||||
Input interface{} // 请求参数结构体 (Query/Path/Body)
|
||||
Output interface{} // 响应参数结构体
|
||||
Tags []string
|
||||
Auth bool // 是否需要认证图标 (预留)
|
||||
Summary string // 简短摘要(中文,一行)
|
||||
Description string // 详细说明,支持 Markdown 语法(可选)
|
||||
Input interface{} // 请求参数结构体 (Query/Path/Body)
|
||||
Output interface{} // 响应参数结构体
|
||||
Tags []string // 分类标签
|
||||
Auth bool // 是否需要认证图标 (预留)
|
||||
FileUploads []FileUploadField // 文件上传字段列表(设置此字段时请求类型为 multipart/form-data)
|
||||
}
|
||||
|
||||
// pathParamRegex 用于匹配 Fiber 的路径参数格式 /:param
|
||||
@@ -33,6 +42,19 @@ func Register(router fiber.Router, doc *openapi.Generator, basePath, method, pat
|
||||
if doc != nil {
|
||||
fullPath := basePath + path
|
||||
openapiPath := pathParamRegex.ReplaceAllString(fullPath, "/{$1}")
|
||||
doc.AddOperation(method, openapiPath, spec.Summary, spec.Input, spec.Output, spec.Auth, spec.Tags...)
|
||||
|
||||
if len(spec.FileUploads) > 0 {
|
||||
fileFields := make([]openapi.FileUploadField, len(spec.FileUploads))
|
||||
for i, f := range spec.FileUploads {
|
||||
fileFields[i] = openapi.FileUploadField{
|
||||
Name: f.Name,
|
||||
Description: f.Description,
|
||||
Required: f.Required,
|
||||
}
|
||||
}
|
||||
doc.AddMultipartOperation(method, openapiPath, spec.Summary, spec.Description, spec.Input, spec.Output, spec.Auth, fileFields, spec.Tags...)
|
||||
} else {
|
||||
doc.AddOperation(method, openapiPath, spec.Summary, spec.Description, spec.Input, spec.Output, spec.Auth, spec.Tags...)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
71
internal/routes/storage.go
Normal file
71
internal/routes/storage.go
Normal file
@@ -0,0 +1,71 @@
|
||||
package routes
|
||||
|
||||
import (
|
||||
"github.com/gofiber/fiber/v2"
|
||||
|
||||
"github.com/break/junhong_cmp_fiber/internal/handler/admin"
|
||||
"github.com/break/junhong_cmp_fiber/internal/model/dto"
|
||||
"github.com/break/junhong_cmp_fiber/pkg/openapi"
|
||||
)
|
||||
|
||||
func registerStorageRoutes(router fiber.Router, handler *admin.StorageHandler, doc *openapi.Generator, basePath string) {
|
||||
storage := router.Group("/storage")
|
||||
groupPath := basePath + "/storage"
|
||||
|
||||
Register(storage, doc, groupPath, "POST", "/upload-url", handler.GetUploadURL, RouteSpec{
|
||||
Summary: "获取文件上传预签名 URL",
|
||||
Description: `## 文件上传流程
|
||||
|
||||
本接口用于获取对象存储的预签名上传 URL,实现前端直传文件到对象存储。
|
||||
|
||||
### 完整流程
|
||||
|
||||
1. **调用本接口** 获取预签名 URL 和 file_key
|
||||
2. **使用预签名 URL 上传文件** 发起 PUT 请求直接上传到对象存储
|
||||
3. **调用业务接口** 使用 file_key 调用相关业务接口(如 ICCID 导入)
|
||||
|
||||
### 前端上传示例
|
||||
|
||||
` + "```" + `javascript
|
||||
// 1. 获取预签名 URL
|
||||
const { data } = await api.post('/storage/upload-url', {
|
||||
file_name: 'cards.csv',
|
||||
content_type: 'text/csv',
|
||||
purpose: 'iot_import'
|
||||
});
|
||||
|
||||
// 2. 上传文件到对象存储
|
||||
await fetch(data.upload_url, {
|
||||
method: 'PUT',
|
||||
headers: { 'Content-Type': 'text/csv' },
|
||||
body: file
|
||||
});
|
||||
|
||||
// 3. 调用业务接口
|
||||
await api.post('/iot-cards/import', {
|
||||
carrier_id: 1,
|
||||
batch_no: 'BATCH-2025-01',
|
||||
file_key: data.file_key
|
||||
});
|
||||
` + "```" + `
|
||||
|
||||
### purpose 可选值
|
||||
|
||||
| 值 | 说明 | 生成路径格式 |
|
||||
|---|------|-------------|
|
||||
| iot_import | ICCID 导入 | imports/YYYY/MM/DD/uuid.csv |
|
||||
| export | 数据导出 | exports/YYYY/MM/DD/uuid.xlsx |
|
||||
| attachment | 附件上传 | attachments/YYYY/MM/DD/uuid.ext |
|
||||
|
||||
### 注意事项
|
||||
|
||||
- 预签名 URL 有效期 **15 分钟**,请及时使用
|
||||
- 上传时 Content-Type 需与请求时一致
|
||||
- file_key 在上传成功后永久有效,用于后续业务接口调用
|
||||
- 上传失败时可重新调用本接口获取新的 URL`,
|
||||
Tags: []string{"对象存储"},
|
||||
Input: new(dto.GetUploadURLRequest),
|
||||
Output: new(dto.GetUploadURLResponse),
|
||||
Auth: true,
|
||||
})
|
||||
}
|
||||
@@ -3,7 +3,7 @@ package iot_card_import
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"io"
|
||||
"path/filepath"
|
||||
"time"
|
||||
|
||||
"github.com/break/junhong_cmp_fiber/internal/model"
|
||||
@@ -14,7 +14,6 @@ import (
|
||||
"github.com/break/junhong_cmp_fiber/pkg/errors"
|
||||
"github.com/break/junhong_cmp_fiber/pkg/middleware"
|
||||
"github.com/break/junhong_cmp_fiber/pkg/queue"
|
||||
"github.com/break/junhong_cmp_fiber/pkg/utils"
|
||||
"gorm.io/gorm"
|
||||
)
|
||||
|
||||
@@ -58,7 +57,7 @@ type IotCardImportPayload struct {
|
||||
TaskID uint `json:"task_id"`
|
||||
}
|
||||
|
||||
func (s *Service) CreateImportTask(ctx context.Context, req *dto.ImportIotCardRequest, csvReader io.Reader, fileName string) (*dto.ImportIotCardResponse, error) {
|
||||
func (s *Service) CreateImportTask(ctx context.Context, req *dto.ImportIotCardRequest) (*dto.ImportIotCardResponse, error) {
|
||||
userID := middleware.GetUserIDFromContext(ctx)
|
||||
if userID == 0 {
|
||||
return nil, errors.New(errors.CodeUnauthorized, "未授权访问")
|
||||
@@ -69,29 +68,17 @@ func (s *Service) CreateImportTask(ctx context.Context, req *dto.ImportIotCardRe
|
||||
return nil, errors.New(errors.CodeInvalidParam, "运营商不存在")
|
||||
}
|
||||
|
||||
parseResult, err := utils.ParseICCIDFromCSV(csvReader)
|
||||
if err != nil {
|
||||
return nil, errors.New(errors.CodeInvalidParam, "CSV 解析失败: "+err.Error())
|
||||
}
|
||||
|
||||
if parseResult.TotalCount == 0 {
|
||||
return nil, errors.New(errors.CodeInvalidParam, "CSV 文件中没有有效的 ICCID")
|
||||
}
|
||||
|
||||
taskNo := s.importTaskStore.GenerateTaskNo(ctx)
|
||||
fileName := filepath.Base(req.FileKey)
|
||||
|
||||
task := &model.IotCardImportTask{
|
||||
TaskNo: taskNo,
|
||||
Status: model.ImportTaskStatusPending,
|
||||
CarrierID: req.CarrierID,
|
||||
CarrierType: carrier.CarrierType,
|
||||
BatchNo: req.BatchNo,
|
||||
FileName: fileName,
|
||||
TotalCount: parseResult.TotalCount,
|
||||
SuccessCount: 0,
|
||||
SkipCount: 0,
|
||||
FailCount: 0,
|
||||
ICCIDList: model.ICCIDListJSON(parseResult.ICCIDs),
|
||||
TaskNo: taskNo,
|
||||
Status: model.ImportTaskStatusPending,
|
||||
CarrierID: req.CarrierID,
|
||||
CarrierType: carrier.CarrierType,
|
||||
BatchNo: req.BatchNo,
|
||||
FileName: fileName,
|
||||
StorageKey: req.FileKey,
|
||||
}
|
||||
task.Creator = userID
|
||||
task.Updater = userID
|
||||
@@ -110,7 +97,7 @@ func (s *Service) CreateImportTask(ctx context.Context, req *dto.ImportIotCardRe
|
||||
return &dto.ImportIotCardResponse{
|
||||
TaskID: task.ID,
|
||||
TaskNo: taskNo,
|
||||
Message: fmt.Sprintf("导入任务已创建,共 %d 条 ICCID 待处理", parseResult.TotalCount),
|
||||
Message: "导入任务已创建,Worker 将异步处理文件",
|
||||
}, nil
|
||||
}
|
||||
|
||||
@@ -194,6 +181,7 @@ func (s *Service) GetByID(ctx context.Context, id uint) (*dto.ImportTaskDetailRe
|
||||
resp.SkippedItems = append(resp.SkippedItems, &dto.ImportResultItemDTO{
|
||||
Line: item.Line,
|
||||
ICCID: item.ICCID,
|
||||
MSISDN: item.MSISDN,
|
||||
Reason: item.Reason,
|
||||
})
|
||||
}
|
||||
@@ -202,6 +190,7 @@ func (s *Service) GetByID(ctx context.Context, id uint) (*dto.ImportTaskDetailRe
|
||||
resp.FailedItems = append(resp.FailedItems, &dto.ImportResultItemDTO{
|
||||
Line: item.Line,
|
||||
ICCID: item.ICCID,
|
||||
MSISDN: item.MSISDN,
|
||||
Reason: item.Reason,
|
||||
})
|
||||
}
|
||||
|
||||
@@ -125,6 +125,15 @@ func (s *IotCardImportTaskStore) List(ctx context.Context, opts *store.QueryOpti
|
||||
return tasks, total, nil
|
||||
}
|
||||
|
||||
func (s *IotCardImportTaskStore) UpdateCardList(ctx context.Context, id uint, cards model.CardListJSON, totalCount int) error {
|
||||
updates := map[string]interface{}{
|
||||
"card_list": cards,
|
||||
"total_count": totalCount,
|
||||
"updated_at": time.Now(),
|
||||
}
|
||||
return s.db.WithContext(ctx).Model(&model.IotCardImportTask{}).Where("id = ?", id).Updates(updates).Error
|
||||
}
|
||||
|
||||
func (s *IotCardImportTaskStore) GenerateTaskNo(ctx context.Context) string {
|
||||
now := time.Now()
|
||||
dateStr := now.Format("20060102")
|
||||
|
||||
@@ -2,6 +2,8 @@ package task
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"os"
|
||||
"time"
|
||||
|
||||
"github.com/bytedance/sonic"
|
||||
@@ -14,9 +16,16 @@ import (
|
||||
"github.com/break/junhong_cmp_fiber/internal/store/postgres"
|
||||
"github.com/break/junhong_cmp_fiber/pkg/constants"
|
||||
pkggorm "github.com/break/junhong_cmp_fiber/pkg/gorm"
|
||||
"github.com/break/junhong_cmp_fiber/pkg/storage"
|
||||
"github.com/break/junhong_cmp_fiber/pkg/utils"
|
||||
"github.com/break/junhong_cmp_fiber/pkg/validator"
|
||||
)
|
||||
|
||||
var (
|
||||
ErrStorageNotConfigured = errors.New("对象存储服务未配置")
|
||||
ErrStorageKeyEmpty = errors.New("文件存储路径为空")
|
||||
)
|
||||
|
||||
const batchSize = 1000
|
||||
|
||||
type IotCardImportPayload struct {
|
||||
@@ -28,15 +37,24 @@ type IotCardImportHandler struct {
|
||||
redis *redis.Client
|
||||
importTaskStore *postgres.IotCardImportTaskStore
|
||||
iotCardStore *postgres.IotCardStore
|
||||
storageService *storage.Service
|
||||
logger *zap.Logger
|
||||
}
|
||||
|
||||
func NewIotCardImportHandler(db *gorm.DB, redis *redis.Client, importTaskStore *postgres.IotCardImportTaskStore, iotCardStore *postgres.IotCardStore, logger *zap.Logger) *IotCardImportHandler {
|
||||
func NewIotCardImportHandler(
|
||||
db *gorm.DB,
|
||||
redis *redis.Client,
|
||||
importTaskStore *postgres.IotCardImportTaskStore,
|
||||
iotCardStore *postgres.IotCardStore,
|
||||
storageSvc *storage.Service,
|
||||
logger *zap.Logger,
|
||||
) *IotCardImportHandler {
|
||||
return &IotCardImportHandler{
|
||||
db: db,
|
||||
redis: redis,
|
||||
importTaskStore: importTaskStore,
|
||||
iotCardStore: iotCardStore,
|
||||
storageService: storageSvc,
|
||||
logger: logger,
|
||||
}
|
||||
}
|
||||
@@ -75,9 +93,23 @@ func (h *IotCardImportHandler) HandleIotCardImport(ctx context.Context, task *as
|
||||
h.logger.Info("开始处理 IoT 卡导入任务",
|
||||
zap.Uint("task_id", importTask.ID),
|
||||
zap.String("task_no", importTask.TaskNo),
|
||||
zap.Int("total_count", importTask.TotalCount),
|
||||
zap.String("storage_key", importTask.StorageKey),
|
||||
)
|
||||
|
||||
cards, totalCount, err := h.downloadAndParseCSV(ctx, importTask)
|
||||
if err != nil {
|
||||
h.logger.Error("下载或解析 CSV 失败",
|
||||
zap.Uint("task_id", importTask.ID),
|
||||
zap.Error(err),
|
||||
)
|
||||
h.importTaskStore.UpdateStatus(ctx, importTask.ID, model.ImportTaskStatusFailed, err.Error())
|
||||
return asynq.SkipRetry
|
||||
}
|
||||
|
||||
importTask.CardList = cards
|
||||
importTask.TotalCount = totalCount
|
||||
h.importTaskStore.UpdateCardList(ctx, importTask.ID, cards, totalCount)
|
||||
|
||||
result := h.processImport(ctx, importTask)
|
||||
|
||||
h.importTaskStore.UpdateResult(ctx, importTask.ID, result.successCount, result.skipCount, result.failCount, result.skippedItems, result.failedItems)
|
||||
@@ -98,6 +130,43 @@ func (h *IotCardImportHandler) HandleIotCardImport(ctx context.Context, task *as
|
||||
return nil
|
||||
}
|
||||
|
||||
func (h *IotCardImportHandler) downloadAndParseCSV(ctx context.Context, task *model.IotCardImportTask) (model.CardListJSON, int, error) {
|
||||
if h.storageService == nil {
|
||||
return nil, 0, ErrStorageNotConfigured
|
||||
}
|
||||
|
||||
if task.StorageKey == "" {
|
||||
return nil, 0, ErrStorageKeyEmpty
|
||||
}
|
||||
|
||||
localPath, cleanup, err := h.storageService.DownloadToTemp(ctx, task.StorageKey)
|
||||
if err != nil {
|
||||
return nil, 0, err
|
||||
}
|
||||
defer cleanup()
|
||||
|
||||
f, err := os.Open(localPath)
|
||||
if err != nil {
|
||||
return nil, 0, err
|
||||
}
|
||||
defer f.Close()
|
||||
|
||||
parseResult, err := utils.ParseCardCSV(f)
|
||||
if err != nil {
|
||||
return nil, 0, err
|
||||
}
|
||||
|
||||
cards := make(model.CardListJSON, 0, len(parseResult.Cards))
|
||||
for _, card := range parseResult.Cards {
|
||||
cards = append(cards, model.CardItem{
|
||||
ICCID: card.ICCID,
|
||||
MSISDN: card.MSISDN,
|
||||
})
|
||||
}
|
||||
|
||||
return cards, parseResult.TotalCount, nil
|
||||
}
|
||||
|
||||
type importResult struct {
|
||||
successCount int
|
||||
skipCount int
|
||||
@@ -112,59 +181,72 @@ func (h *IotCardImportHandler) processImport(ctx context.Context, task *model.Io
|
||||
failedItems: make(model.ImportResultItems, 0),
|
||||
}
|
||||
|
||||
iccids := h.getICCIDsFromTask(task)
|
||||
if len(iccids) == 0 {
|
||||
cards := h.getCardsFromTask(task)
|
||||
if len(cards) == 0 {
|
||||
return result
|
||||
}
|
||||
|
||||
for i := 0; i < len(iccids); i += batchSize {
|
||||
end := min(i+batchSize, len(iccids))
|
||||
batch := iccids[i:end]
|
||||
for i := 0; i < len(cards); i += batchSize {
|
||||
end := min(i+batchSize, len(cards))
|
||||
batch := cards[i:end]
|
||||
h.processBatch(ctx, task, batch, i+1, result)
|
||||
}
|
||||
|
||||
return result
|
||||
}
|
||||
|
||||
func (h *IotCardImportHandler) getICCIDsFromTask(task *model.IotCardImportTask) []string {
|
||||
return []string(task.ICCIDList)
|
||||
// getCardsFromTask 从任务中获取待导入的卡列表
|
||||
func (h *IotCardImportHandler) getCardsFromTask(task *model.IotCardImportTask) []model.CardItem {
|
||||
return []model.CardItem(task.CardList)
|
||||
}
|
||||
|
||||
func (h *IotCardImportHandler) processBatch(ctx context.Context, task *model.IotCardImportTask, batch []string, startLine int, result *importResult) {
|
||||
validICCIDs := make([]string, 0)
|
||||
lineMap := make(map[string]int)
|
||||
func (h *IotCardImportHandler) processBatch(ctx context.Context, task *model.IotCardImportTask, batch []model.CardItem, startLine int, result *importResult) {
|
||||
type cardMeta struct {
|
||||
line int
|
||||
msisdn string
|
||||
}
|
||||
validCards := make([]model.CardItem, 0)
|
||||
cardMetaMap := make(map[string]cardMeta)
|
||||
|
||||
for i, iccid := range batch {
|
||||
for i, card := range batch {
|
||||
line := startLine + i
|
||||
lineMap[iccid] = line
|
||||
cardMetaMap[card.ICCID] = cardMeta{line: line, msisdn: card.MSISDN}
|
||||
|
||||
validationResult := validator.ValidateICCID(iccid, task.CarrierType)
|
||||
validationResult := validator.ValidateICCID(card.ICCID, task.CarrierType)
|
||||
if !validationResult.Valid {
|
||||
result.failedItems = append(result.failedItems, model.ImportResultItem{
|
||||
Line: line,
|
||||
ICCID: iccid,
|
||||
ICCID: card.ICCID,
|
||||
MSISDN: card.MSISDN,
|
||||
Reason: validationResult.Message,
|
||||
})
|
||||
result.failCount++
|
||||
continue
|
||||
}
|
||||
validICCIDs = append(validICCIDs, iccid)
|
||||
validCards = append(validCards, card)
|
||||
}
|
||||
|
||||
if len(validICCIDs) == 0 {
|
||||
if len(validCards) == 0 {
|
||||
return
|
||||
}
|
||||
|
||||
validICCIDs := make([]string, len(validCards))
|
||||
for i, card := range validCards {
|
||||
validICCIDs[i] = card.ICCID
|
||||
}
|
||||
|
||||
existingMap, err := h.iotCardStore.ExistsByICCIDBatch(ctx, validICCIDs)
|
||||
if err != nil {
|
||||
h.logger.Error("批量检查 ICCID 是否存在失败",
|
||||
zap.Error(err),
|
||||
zap.Int("batch_size", len(validICCIDs)),
|
||||
)
|
||||
for _, iccid := range validICCIDs {
|
||||
for _, card := range validCards {
|
||||
meta := cardMetaMap[card.ICCID]
|
||||
result.failedItems = append(result.failedItems, model.ImportResultItem{
|
||||
Line: lineMap[iccid],
|
||||
ICCID: iccid,
|
||||
Line: meta.line,
|
||||
ICCID: card.ICCID,
|
||||
MSISDN: meta.msisdn,
|
||||
Reason: "数据库查询失败",
|
||||
})
|
||||
result.failCount++
|
||||
@@ -172,29 +254,32 @@ func (h *IotCardImportHandler) processBatch(ctx context.Context, task *model.Iot
|
||||
return
|
||||
}
|
||||
|
||||
newICCIDs := make([]string, 0)
|
||||
for _, iccid := range validICCIDs {
|
||||
if existingMap[iccid] {
|
||||
newCards := make([]model.CardItem, 0)
|
||||
for _, card := range validCards {
|
||||
meta := cardMetaMap[card.ICCID]
|
||||
if existingMap[card.ICCID] {
|
||||
result.skippedItems = append(result.skippedItems, model.ImportResultItem{
|
||||
Line: lineMap[iccid],
|
||||
ICCID: iccid,
|
||||
Line: meta.line,
|
||||
ICCID: card.ICCID,
|
||||
MSISDN: meta.msisdn,
|
||||
Reason: "ICCID 已存在",
|
||||
})
|
||||
result.skipCount++
|
||||
} else {
|
||||
newICCIDs = append(newICCIDs, iccid)
|
||||
newCards = append(newCards, card)
|
||||
}
|
||||
}
|
||||
|
||||
if len(newICCIDs) == 0 {
|
||||
if len(newCards) == 0 {
|
||||
return
|
||||
}
|
||||
|
||||
cards := make([]*model.IotCard, 0, len(newICCIDs))
|
||||
iotCards := make([]*model.IotCard, 0, len(newCards))
|
||||
now := time.Now()
|
||||
for _, iccid := range newICCIDs {
|
||||
card := &model.IotCard{
|
||||
ICCID: iccid,
|
||||
for _, card := range newCards {
|
||||
iotCard := &model.IotCard{
|
||||
ICCID: card.ICCID,
|
||||
MSISDN: card.MSISDN,
|
||||
CarrierID: task.CarrierID,
|
||||
BatchNo: task.BatchNo,
|
||||
Status: constants.IotCardStatusInStock,
|
||||
@@ -203,22 +288,24 @@ func (h *IotCardImportHandler) processBatch(ctx context.Context, task *model.Iot
|
||||
RealNameStatus: constants.RealNameStatusNotVerified,
|
||||
NetworkStatus: constants.NetworkStatusOffline,
|
||||
}
|
||||
card.BaseModel.Creator = task.Creator
|
||||
card.BaseModel.Updater = task.Creator
|
||||
card.CreatedAt = now
|
||||
card.UpdatedAt = now
|
||||
cards = append(cards, card)
|
||||
iotCard.BaseModel.Creator = task.Creator
|
||||
iotCard.BaseModel.Updater = task.Creator
|
||||
iotCard.CreatedAt = now
|
||||
iotCard.UpdatedAt = now
|
||||
iotCards = append(iotCards, iotCard)
|
||||
}
|
||||
|
||||
if err := h.iotCardStore.CreateBatch(ctx, cards); err != nil {
|
||||
if err := h.iotCardStore.CreateBatch(ctx, iotCards); err != nil {
|
||||
h.logger.Error("批量创建 IoT 卡失败",
|
||||
zap.Error(err),
|
||||
zap.Int("batch_size", len(cards)),
|
||||
zap.Int("batch_size", len(iotCards)),
|
||||
)
|
||||
for _, iccid := range newICCIDs {
|
||||
for _, card := range newCards {
|
||||
meta := cardMetaMap[card.ICCID]
|
||||
result.failedItems = append(result.failedItems, model.ImportResultItem{
|
||||
Line: lineMap[iccid],
|
||||
ICCID: iccid,
|
||||
Line: meta.line,
|
||||
ICCID: card.ICCID,
|
||||
MSISDN: meta.msisdn,
|
||||
Reason: "数据库写入失败",
|
||||
})
|
||||
result.failCount++
|
||||
@@ -226,5 +313,5 @@ func (h *IotCardImportHandler) processBatch(ctx context.Context, task *model.Iot
|
||||
return
|
||||
}
|
||||
|
||||
result.successCount += len(newICCIDs)
|
||||
result.successCount += len(newCards)
|
||||
}
|
||||
|
||||
@@ -22,7 +22,7 @@ func TestIotCardImportHandler_ProcessImport(t *testing.T) {
|
||||
importTaskStore := postgres.NewIotCardImportTaskStore(tx, rdb)
|
||||
iotCardStore := postgres.NewIotCardStore(tx, rdb)
|
||||
|
||||
handler := NewIotCardImportHandler(tx, rdb, importTaskStore, iotCardStore, logger)
|
||||
handler := NewIotCardImportHandler(tx, rdb, importTaskStore, iotCardStore, nil, logger)
|
||||
ctx := context.Background()
|
||||
|
||||
t.Run("成功导入新ICCID", func(t *testing.T) {
|
||||
@@ -30,8 +30,12 @@ func TestIotCardImportHandler_ProcessImport(t *testing.T) {
|
||||
CarrierID: 1,
|
||||
CarrierType: constants.CarrierCodeCMCC,
|
||||
BatchNo: "TEST_BATCH_001",
|
||||
ICCIDList: model.ICCIDListJSON{"89860012345678905001", "89860012345678905002", "89860012345678905003"},
|
||||
TotalCount: 3,
|
||||
CardList: model.CardListJSON{
|
||||
{ICCID: "89860012345678905001", MSISDN: "13800000001"},
|
||||
{ICCID: "89860012345678905002", MSISDN: "13800000002"},
|
||||
{ICCID: "89860012345678905003", MSISDN: "13800000003"},
|
||||
},
|
||||
TotalCount: 3,
|
||||
}
|
||||
task.Creator = 1
|
||||
|
||||
@@ -43,6 +47,9 @@ func TestIotCardImportHandler_ProcessImport(t *testing.T) {
|
||||
|
||||
exists, _ := iotCardStore.ExistsByICCID(ctx, "89860012345678905001")
|
||||
assert.True(t, exists)
|
||||
|
||||
card, _ := iotCardStore.GetByICCID(ctx, "89860012345678905001")
|
||||
assert.Equal(t, "13800000001", card.MSISDN)
|
||||
})
|
||||
|
||||
t.Run("跳过已存在的ICCID", func(t *testing.T) {
|
||||
@@ -58,8 +65,11 @@ func TestIotCardImportHandler_ProcessImport(t *testing.T) {
|
||||
CarrierID: 1,
|
||||
CarrierType: constants.CarrierCodeCMCC,
|
||||
BatchNo: "TEST_BATCH_002",
|
||||
ICCIDList: model.ICCIDListJSON{"89860012345678906001", "89860012345678906002"},
|
||||
TotalCount: 2,
|
||||
CardList: model.CardListJSON{
|
||||
{ICCID: "89860012345678906001", MSISDN: "13800000011"},
|
||||
{ICCID: "89860012345678906002", MSISDN: "13800000012"},
|
||||
},
|
||||
TotalCount: 2,
|
||||
}
|
||||
task.Creator = 1
|
||||
|
||||
@@ -70,6 +80,7 @@ func TestIotCardImportHandler_ProcessImport(t *testing.T) {
|
||||
assert.Equal(t, 0, result.failCount)
|
||||
assert.Len(t, result.skippedItems, 1)
|
||||
assert.Equal(t, "89860012345678906001", result.skippedItems[0].ICCID)
|
||||
assert.Equal(t, "13800000011", result.skippedItems[0].MSISDN)
|
||||
assert.Equal(t, "ICCID 已存在", result.skippedItems[0].Reason)
|
||||
})
|
||||
|
||||
@@ -78,8 +89,11 @@ func TestIotCardImportHandler_ProcessImport(t *testing.T) {
|
||||
CarrierID: 1,
|
||||
CarrierType: constants.CarrierCodeCTCC,
|
||||
BatchNo: "TEST_BATCH_003",
|
||||
ICCIDList: model.ICCIDListJSON{"89860312345678907001", "898603123456789070"},
|
||||
TotalCount: 2,
|
||||
CardList: model.CardListJSON{
|
||||
{ICCID: "89860312345678907001", MSISDN: "13900000001"},
|
||||
{ICCID: "898603123456789070", MSISDN: "13900000002"},
|
||||
},
|
||||
TotalCount: 2,
|
||||
}
|
||||
task.Creator = 1
|
||||
|
||||
@@ -89,6 +103,7 @@ func TestIotCardImportHandler_ProcessImport(t *testing.T) {
|
||||
assert.Equal(t, 0, result.skipCount)
|
||||
assert.Equal(t, 2, result.failCount)
|
||||
assert.Len(t, result.failedItems, 2)
|
||||
assert.Equal(t, "13900000001", result.failedItems[0].MSISDN)
|
||||
})
|
||||
|
||||
t.Run("混合场景-成功跳过和失败", func(t *testing.T) {
|
||||
@@ -104,10 +119,10 @@ func TestIotCardImportHandler_ProcessImport(t *testing.T) {
|
||||
CarrierID: 1,
|
||||
CarrierType: constants.CarrierCodeCMCC,
|
||||
BatchNo: "TEST_BATCH_004",
|
||||
ICCIDList: model.ICCIDListJSON{
|
||||
"89860012345678908001",
|
||||
"89860012345678908002",
|
||||
"invalid!iccid",
|
||||
CardList: model.CardListJSON{
|
||||
{ICCID: "89860012345678908001", MSISDN: "13800000021"},
|
||||
{ICCID: "89860012345678908002", MSISDN: "13800000022"},
|
||||
{ICCID: "invalid!iccid", MSISDN: "13800000023"},
|
||||
},
|
||||
TotalCount: 3,
|
||||
}
|
||||
@@ -120,12 +135,12 @@ func TestIotCardImportHandler_ProcessImport(t *testing.T) {
|
||||
assert.Equal(t, 1, result.failCount)
|
||||
})
|
||||
|
||||
t.Run("空ICCID列表", func(t *testing.T) {
|
||||
t.Run("空卡列表", func(t *testing.T) {
|
||||
task := &model.IotCardImportTask{
|
||||
CarrierID: 1,
|
||||
CarrierType: constants.CarrierCodeCMCC,
|
||||
BatchNo: "TEST_BATCH_005",
|
||||
ICCIDList: model.ICCIDListJSON{},
|
||||
CardList: model.CardListJSON{},
|
||||
TotalCount: 0,
|
||||
}
|
||||
|
||||
@@ -146,10 +161,10 @@ func TestIotCardImportHandler_ProcessBatch(t *testing.T) {
|
||||
importTaskStore := postgres.NewIotCardImportTaskStore(tx, rdb)
|
||||
iotCardStore := postgres.NewIotCardStore(tx, rdb)
|
||||
|
||||
handler := NewIotCardImportHandler(tx, rdb, importTaskStore, iotCardStore, logger)
|
||||
handler := NewIotCardImportHandler(tx, rdb, importTaskStore, iotCardStore, nil, logger)
|
||||
ctx := context.Background()
|
||||
|
||||
t.Run("验证行号正确记录", func(t *testing.T) {
|
||||
t.Run("验证行号和MSISDN正确记录", func(t *testing.T) {
|
||||
existingCard := &model.IotCard{
|
||||
ICCID: "89860012345678909002",
|
||||
CardType: "data_card",
|
||||
@@ -165,10 +180,10 @@ func TestIotCardImportHandler_ProcessBatch(t *testing.T) {
|
||||
}
|
||||
task.Creator = 1
|
||||
|
||||
batch := []string{
|
||||
"89860012345678909001",
|
||||
"89860012345678909002",
|
||||
"invalid",
|
||||
batch := []model.CardItem{
|
||||
{ICCID: "89860012345678909001", MSISDN: "13800000031"},
|
||||
{ICCID: "89860012345678909002", MSISDN: "13800000032"},
|
||||
{ICCID: "invalid", MSISDN: "13800000033"},
|
||||
}
|
||||
result := &importResult{
|
||||
skippedItems: make(model.ImportResultItems, 0),
|
||||
@@ -182,6 +197,8 @@ func TestIotCardImportHandler_ProcessBatch(t *testing.T) {
|
||||
assert.Equal(t, 1, result.failCount)
|
||||
|
||||
assert.Equal(t, 101, result.skippedItems[0].Line)
|
||||
assert.Equal(t, "13800000032", result.skippedItems[0].MSISDN)
|
||||
assert.Equal(t, 102, result.failedItems[0].Line)
|
||||
assert.Equal(t, "13800000033", result.failedItems[0].MSISDN)
|
||||
})
|
||||
}
|
||||
|
||||
2
migrations/000015_add_card_list_to_import_task.down.sql
Normal file
2
migrations/000015_add_card_list_to_import_task.down.sql
Normal file
@@ -0,0 +1,2 @@
|
||||
ALTER TABLE tb_iot_card_import_task
|
||||
DROP COLUMN IF EXISTS card_list;
|
||||
5
migrations/000015_add_card_list_to_import_task.up.sql
Normal file
5
migrations/000015_add_card_list_to_import_task.up.sql
Normal file
@@ -0,0 +1,5 @@
|
||||
-- 添加 card_list 字段存储 ICCID + MSISDN 列表
|
||||
ALTER TABLE tb_iot_card_import_task
|
||||
ADD COLUMN card_list JSONB DEFAULT '[]';
|
||||
|
||||
COMMENT ON COLUMN tb_iot_card_import_task.card_list IS '待导入卡列表[{iccid, msisdn}]';
|
||||
@@ -0,0 +1,3 @@
|
||||
ALTER TABLE tb_iot_card_import_task
|
||||
DROP COLUMN IF EXISTS storage_bucket,
|
||||
DROP COLUMN IF EXISTS storage_key;
|
||||
@@ -0,0 +1,6 @@
|
||||
ALTER TABLE tb_iot_card_import_task
|
||||
ADD COLUMN IF NOT EXISTS storage_bucket VARCHAR(100),
|
||||
ADD COLUMN IF NOT EXISTS storage_key VARCHAR(500);
|
||||
|
||||
COMMENT ON COLUMN tb_iot_card_import_task.storage_bucket IS '对象存储桶名';
|
||||
COMMENT ON COLUMN tb_iot_card_import_task.storage_key IS '对象存储文件路径';
|
||||
@@ -0,0 +1,2 @@
|
||||
schema: spec-driven
|
||||
created: 2026-01-24
|
||||
233
openspec/changes/archive/2026-01-24-add-object-storage/design.md
Normal file
233
openspec/changes/archive/2026-01-24-add-object-storage/design.md
Normal file
@@ -0,0 +1,233 @@
|
||||
# 通用对象存储 - 技术设计
|
||||
|
||||
## Context
|
||||
|
||||
### 背景
|
||||
|
||||
当前系统的 ICCID 导入功能采用传统的文件上传方式:前端上传 CSV 文件到后端,后端解析后处理。这种方式存在以下问题:
|
||||
|
||||
1. **性能瓶颈**:大文件上传占用后端带宽和内存
|
||||
2. **扩展性差**:未来导出功能也需要文件存储能力
|
||||
3. **安全风险**:文件处理在内存中进行,存在 OOM 风险
|
||||
|
||||
### 现状
|
||||
|
||||
- 联通云对象存储(CUCloud OSS)已开通,Bucket `cmp` 已创建
|
||||
- 联通云 OSS 兼容 AWS S3 API,支持预签名 URL(已验证)
|
||||
- 项目使用 `github.com/aws/aws-sdk-go` v1 版本
|
||||
|
||||
### 约束
|
||||
|
||||
- 本项目作为公司后端模板,设计需要通用化
|
||||
- 联通云 OSS 的 Endpoint 格式:`http://obs-{region}.cucloud.cn`
|
||||
- 同一时刻只使用一个云存储提供商(不需要多云并存)
|
||||
|
||||
## Goals / Non-Goals
|
||||
|
||||
### Goals
|
||||
|
||||
1. **通用对象存储能力**:提供可复用的对象存储包 `pkg/storage/`
|
||||
2. **预签名 URL 支持**:前端直传,不经过后端
|
||||
3. **ICCID 导入改造**:集成对象存储,提升性能
|
||||
4. **配置驱动**:通过配置文件切换不同云存储
|
||||
|
||||
### Non-Goals
|
||||
|
||||
1. **不实现导出功能**:只准备能力,导出功能后续单独开发
|
||||
2. **不实现多云并存**:同一时刻只用一个云
|
||||
3. **不删除对象存储文件**:导入完成后只删除本地临时文件
|
||||
4. **不实现断点续传**:小文件(CSV)不需要
|
||||
|
||||
## Decisions
|
||||
|
||||
### Decision 1: 使用 AWS SDK v1 而非 v2
|
||||
|
||||
**选择**:`github.com/aws/aws-sdk-go`(v1)
|
||||
|
||||
**理由**:
|
||||
- 联通云官方文档推荐使用 v1
|
||||
- 已验证 v1 在联通云上的预签名功能正常工作
|
||||
- v1 的 API 更简洁,学习成本低
|
||||
|
||||
**备选方案**:
|
||||
- AWS SDK v2:API 更现代,但联通云文档无示例,兼容性未知
|
||||
- MinIO Go Client:功能更丰富,但增加额外依赖
|
||||
|
||||
### Decision 2: Provider 接口设计
|
||||
|
||||
**选择**:定义简洁的 `Provider` 接口
|
||||
|
||||
```go
|
||||
type Provider interface {
|
||||
// 上传文件
|
||||
Upload(ctx context.Context, key string, reader io.Reader, contentType string) error
|
||||
// 下载文件到 io.Writer
|
||||
Download(ctx context.Context, key string, writer io.Writer) error
|
||||
// 下载文件到本地临时文件
|
||||
DownloadToTemp(ctx context.Context, key string) (localPath string, cleanup func(), err error)
|
||||
// 删除文件
|
||||
Delete(ctx context.Context, key string) error
|
||||
// 检查文件是否存在
|
||||
Exists(ctx context.Context, key string) (bool, error)
|
||||
// 生成上传预签名 URL
|
||||
GetUploadURL(ctx context.Context, key string, contentType string, expires time.Duration) (string, error)
|
||||
// 生成下载预签名 URL
|
||||
GetDownloadURL(ctx context.Context, key string, expires time.Duration) (string, error)
|
||||
}
|
||||
```
|
||||
|
||||
**理由**:
|
||||
- 接口方法覆盖导入导出所需的全部操作
|
||||
- `DownloadToTemp` 封装临时文件管理,调用者无需关心清理
|
||||
- 不暴露 Bucket 参数,由实现内部管理(配置驱动)
|
||||
|
||||
### Decision 3: 文件路径规范
|
||||
|
||||
**选择**:`{purpose}/{year}/{month}/{day}/{uuid}.{ext}`
|
||||
|
||||
```
|
||||
imports/2025/01/24/550e8400-e29b-41d4.csv
|
||||
exports/2025/01/24/123456-cards.xlsx
|
||||
attachments/2025/01/24/license.pdf
|
||||
```
|
||||
|
||||
**理由**:
|
||||
- 按日期组织便于管理和清理
|
||||
- UUID 保证唯一性
|
||||
- purpose 前缀区分业务场景
|
||||
|
||||
### Decision 4: 配置结构
|
||||
|
||||
**选择**:嵌套配置,支持多种预签名有效期
|
||||
|
||||
```yaml
|
||||
storage:
|
||||
provider: "s3"
|
||||
s3:
|
||||
endpoint: "http://obs-helf.cucloud.cn"
|
||||
region: "cn-langfang-2"
|
||||
bucket: "cmp"
|
||||
access_key_id: "${OSS_ACCESS_KEY_ID}"
|
||||
secret_access_key: "${OSS_SECRET_ACCESS_KEY}"
|
||||
use_ssl: false
|
||||
path_style: true
|
||||
presign:
|
||||
upload_expires: "15m"
|
||||
download_expires: "24h"
|
||||
temp_dir: "/tmp/junhong-storage"
|
||||
```
|
||||
|
||||
**理由**:
|
||||
- 凭证通过环境变量注入,不硬编码
|
||||
- 预签名有效期可配置,适应不同场景
|
||||
- `path_style: true` 确保联通云兼容性
|
||||
|
||||
### Decision 5: Service 层封装
|
||||
|
||||
**选择**:创建 `StorageService` 封装业务逻辑
|
||||
|
||||
```go
|
||||
type StorageService struct {
|
||||
provider Provider
|
||||
config *config.StorageConfig
|
||||
}
|
||||
|
||||
// 获取上传 URL(自动生成 file_key)
|
||||
func (s *StorageService) GetUploadURL(ctx context.Context, purpose, fileName string) (*PresignResult, error)
|
||||
|
||||
// 下载到临时文件(自动清理)
|
||||
func (s *StorageService) DownloadToTemp(ctx context.Context, fileKey string) (string, func(), error)
|
||||
```
|
||||
|
||||
**理由**:
|
||||
- 封装 file_key 生成逻辑(日期 + UUID)
|
||||
- 统一管理临时文件清理
|
||||
- Handler 层只关心业务参数
|
||||
|
||||
### Decision 6: 导入接口改造
|
||||
|
||||
**选择**:移除文件上传,改为传递 file_key
|
||||
|
||||
**Before**:
|
||||
```
|
||||
POST /api/admin/iot-cards/import
|
||||
Content-Type: multipart/form-data
|
||||
carrier_id, batch_no, file
|
||||
```
|
||||
|
||||
**After**:
|
||||
```
|
||||
POST /api/admin/iot-cards/import
|
||||
Content-Type: application/json
|
||||
{
|
||||
"carrier_id": 1,
|
||||
"batch_no": "BATCH-2025-01",
|
||||
"file_key": "imports/2025/01/24/abc123.csv"
|
||||
}
|
||||
```
|
||||
|
||||
**理由**:
|
||||
- JSON 接口更简洁
|
||||
- 文件已在对象存储,只需传路径
|
||||
- Worker 从对象存储下载处理
|
||||
|
||||
## Risks / Trade-offs
|
||||
|
||||
| 风险 | 影响 | 缓解措施 |
|
||||
|------|------|----------|
|
||||
| 联通云服务不可用 | 无法上传/下载文件 | 1) 配置超时和重试 2) 监控告警 |
|
||||
| 预签名 URL 泄露 | 文件可能被非法访问 | 1) 短有效期(15分钟) 2) 使用 HTTPS |
|
||||
| 临时文件未清理 | 磁盘空间占用 | 1) defer cleanup() 2) 定期清理任务 |
|
||||
| **BREAKING** 接口变更 | 前端需要适配 | 1) 与前端团队同步 2) 提供迁移文档 |
|
||||
|
||||
## 包结构
|
||||
|
||||
```
|
||||
pkg/storage/
|
||||
├── storage.go # Provider 接口定义
|
||||
├── types.go # 公共类型(PresignResult, Config)
|
||||
├── s3.go # S3 兼容实现
|
||||
└── service.go # StorageService 封装
|
||||
|
||||
internal/service/storage/
|
||||
└── service.go # 业务层 Service(可选,如需更多业务逻辑)
|
||||
|
||||
internal/handler/admin/
|
||||
└── storage.go # StorageHandler(获取上传 URL)
|
||||
```
|
||||
|
||||
## Migration Plan
|
||||
|
||||
### 部署步骤
|
||||
|
||||
1. **配置准备**:
|
||||
- 在各环境配置文件中添加 `storage` 配置块
|
||||
- 设置环境变量 `OSS_ACCESS_KEY_ID` 和 `OSS_SECRET_ACCESS_KEY`
|
||||
|
||||
2. **数据库迁移**:
|
||||
- 执行迁移添加 `storage_bucket`、`storage_key` 字段
|
||||
|
||||
3. **代码部署**:
|
||||
- 部署新版本后端代码
|
||||
|
||||
4. **前端适配**:
|
||||
- 前端发布新版本,使用新的上传流程
|
||||
|
||||
### 回滚策略
|
||||
|
||||
- 数据库字段为可空,不影响回滚
|
||||
- 旧版前端可继续使用(需保留旧接口一段时间,或不回滚)
|
||||
|
||||
## Open Questions
|
||||
|
||||
1. **是否需要文件大小限制?**
|
||||
- 建议:CSV 文件限制 10MB
|
||||
- 待确认:具体限制值
|
||||
|
||||
2. **是否需要文件类型校验?**
|
||||
- 建议:只允许 `.csv` 文件
|
||||
- 待确认:是否需要更严格的校验
|
||||
|
||||
3. **旧接口保留多久?**
|
||||
- 建议:不保留,直接切换
|
||||
- 待确认:与前端团队协调
|
||||
@@ -0,0 +1,97 @@
|
||||
# 通用对象存储能力
|
||||
|
||||
## Why
|
||||
|
||||
当前 ICCID 导入功能通过后端接收上传文件,占用服务器带宽和内存,大文件处理效率低。同时,未来的导出功能也需要文件存储能力。需要接入联通云对象存储(S3 兼容),采用预签名 URL 方案实现前端直传,提升性能和安全性。
|
||||
|
||||
此外,本项目作为公司后端模板项目,对象存储应设计为通用能力,方便复用到其他系统。
|
||||
|
||||
## What Changes
|
||||
|
||||
- **新增通用对象存储包**:`pkg/storage/` 提供 S3 兼容的对象存储能力
|
||||
- 支持上传、下载、删除、检查存在性
|
||||
- 支持生成预签名上传/下载 URL
|
||||
- 可扩展支持多云(阿里云、腾讯云等,仅需更换 Endpoint)
|
||||
|
||||
- **新增存储 API 接口**:供前端获取预签名上传 URL
|
||||
- `POST /api/admin/storage/upload-url`:获取上传预签名 URL
|
||||
|
||||
- **改造 ICCID 导入流程**:
|
||||
- 移除原有的文件上传处理(`c.FormFile`)
|
||||
- 改为接收 `file_key` 参数(对象存储路径)
|
||||
- Worker 从对象存储下载文件后处理
|
||||
- 处理完成后删除本地临时文件
|
||||
|
||||
- **数据模型变更**:
|
||||
- `IotCardImportTask` 新增 `storage_bucket`、`storage_key` 字段
|
||||
- 需要数据库迁移
|
||||
|
||||
- **配置结构扩展**:
|
||||
- 新增 `storage` 配置块(endpoint、region、bucket、credentials)
|
||||
|
||||
## Capabilities
|
||||
|
||||
### New Capabilities
|
||||
|
||||
- `object-storage`: 通用对象存储能力,提供 S3 兼容的文件上传、下载、删除、预签名 URL 生成功能
|
||||
|
||||
### Modified Capabilities
|
||||
|
||||
- `iot-card-import`: ICCID 导入流程改造,从直接上传改为对象存储集成
|
||||
|
||||
## Impact
|
||||
|
||||
### 代码变更
|
||||
|
||||
| 层级 | 变更内容 |
|
||||
|------|----------|
|
||||
| `pkg/storage/` | 新增:Provider 接口、S3 实现、配置类型 |
|
||||
| `pkg/config/` | 修改:新增 StorageConfig 结构 |
|
||||
| `configs/` | 修改:新增 storage 配置块 |
|
||||
| `internal/bootstrap/` | 修改:初始化 Storage Provider |
|
||||
| `internal/handler/admin/` | 新增:StorageHandler(获取上传 URL)|
|
||||
| `internal/handler/admin/` | 修改:IotCardImportHandler(移除文件上传)|
|
||||
| `internal/service/iot_card_import/` | 修改:接收 file_key 而非文件流 |
|
||||
| `internal/task/` | 修改:从对象存储下载文件处理 |
|
||||
| `internal/model/` | 修改:IotCardImportTask 新增字段 |
|
||||
| `internal/routes/` | 修改:新增 storage 路由 |
|
||||
| `migrations/` | 新增:添加 storage 字段迁移 |
|
||||
|
||||
### API 变更
|
||||
|
||||
| 接口 | 变更类型 | 说明 |
|
||||
|------|----------|------|
|
||||
| `POST /api/admin/storage/upload-url` | 新增 | 获取预签名上传 URL |
|
||||
| `POST /api/admin/iot-cards/import` | **BREAKING** | 移除文件上传,改为传 file_key |
|
||||
|
||||
### 依赖变更
|
||||
|
||||
| 依赖 | 说明 |
|
||||
|------|------|
|
||||
| `github.com/aws/aws-sdk-go` | 新增:AWS S3 兼容 SDK(已验证联通云支持) |
|
||||
|
||||
### 配置变更
|
||||
|
||||
```yaml
|
||||
storage:
|
||||
provider: "s3"
|
||||
s3:
|
||||
endpoint: "http://obs-helf.cucloud.cn"
|
||||
region: "cn-langfang-2"
|
||||
bucket: "cmp"
|
||||
access_key_id: "${OSS_ACCESS_KEY_ID}"
|
||||
secret_access_key: "${OSS_SECRET_ACCESS_KEY}"
|
||||
use_ssl: false
|
||||
path_style: true
|
||||
presign:
|
||||
upload_expires: "15m" # 上传预签名有效期
|
||||
download_expires: "24h" # 下载预签名有效期
|
||||
temp_dir: "/tmp/junhong-storage" # 临时文件目录
|
||||
```
|
||||
|
||||
### 前端适配
|
||||
|
||||
前端需要配合修改上传流程:
|
||||
1. 先调用 `POST /api/admin/storage/upload-url` 获取预签名 URL
|
||||
2. 直接 PUT 到预签名 URL 上传文件
|
||||
3. 上传成功后调用导入接口,传入 `file_key`
|
||||
@@ -0,0 +1,217 @@
|
||||
# 对象存储能力规格
|
||||
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Provider 接口
|
||||
|
||||
系统 SHALL 提供统一的对象存储 Provider 接口,支持 S3 兼容的对象存储服务。
|
||||
|
||||
接口定义:
|
||||
```go
|
||||
type Provider interface {
|
||||
Upload(ctx context.Context, key string, reader io.Reader, contentType string) error
|
||||
Download(ctx context.Context, key string, writer io.Writer) error
|
||||
DownloadToTemp(ctx context.Context, key string) (localPath string, cleanup func(), err error)
|
||||
Delete(ctx context.Context, key string) error
|
||||
Exists(ctx context.Context, key string) (bool, error)
|
||||
GetUploadURL(ctx context.Context, key string, contentType string, expires time.Duration) (string, error)
|
||||
GetDownloadURL(ctx context.Context, key string, expires time.Duration) (string, error)
|
||||
}
|
||||
```
|
||||
|
||||
#### Scenario: 创建 S3 Provider
|
||||
- **WHEN** 系统启动时读取 storage 配置
|
||||
- **THEN** 系统 SHALL 创建 S3Provider 实例并验证连接
|
||||
|
||||
#### Scenario: 配置缺失
|
||||
- **WHEN** storage 配置未设置或不完整
|
||||
- **THEN** 系统 SHALL 记录警告日志并跳过初始化(不影响启动)
|
||||
|
||||
---
|
||||
|
||||
### Requirement: 文件上传
|
||||
|
||||
系统 SHALL 支持通过 Provider 接口上传文件到对象存储。
|
||||
|
||||
#### Scenario: 上传成功
|
||||
- **WHEN** 调用 `Upload(ctx, "imports/test.csv", reader, "text/csv")`
|
||||
- **THEN** 文件 SHALL 被上传到配置的 Bucket 中指定路径
|
||||
- **THEN** 方法 SHALL 返回 nil
|
||||
|
||||
#### Scenario: 上传失败
|
||||
- **WHEN** 对象存储服务不可用
|
||||
- **THEN** 方法 SHALL 返回包含错误详情的 error
|
||||
|
||||
---
|
||||
|
||||
### Requirement: 文件下载
|
||||
|
||||
系统 SHALL 支持从对象存储下载文件。
|
||||
|
||||
#### Scenario: 下载到 Writer
|
||||
- **WHEN** 调用 `Download(ctx, "imports/test.csv", writer)`
|
||||
- **THEN** 文件内容 SHALL 被写入到提供的 writer
|
||||
|
||||
#### Scenario: 下载到临时文件
|
||||
- **WHEN** 调用 `DownloadToTemp(ctx, "imports/test.csv")`
|
||||
- **THEN** 系统 SHALL 下载文件到临时目录
|
||||
- **THEN** 方法 SHALL 返回本地文件路径和 cleanup 函数
|
||||
- **THEN** 调用 cleanup() 后临时文件 SHALL 被删除
|
||||
|
||||
#### Scenario: 文件不存在
|
||||
- **WHEN** 下载的文件在对象存储中不存在
|
||||
- **THEN** 方法 SHALL 返回 "文件不存在" 错误
|
||||
|
||||
---
|
||||
|
||||
### Requirement: 文件删除
|
||||
|
||||
系统 SHALL 支持从对象存储删除文件。
|
||||
|
||||
#### Scenario: 删除成功
|
||||
- **WHEN** 调用 `Delete(ctx, "imports/test.csv")`
|
||||
- **THEN** 文件 SHALL 从对象存储中删除
|
||||
- **THEN** 方法 SHALL 返回 nil
|
||||
|
||||
#### Scenario: 删除不存在的文件
|
||||
- **WHEN** 删除的文件不存在
|
||||
- **THEN** 方法 SHALL 返回 nil(幂等操作)
|
||||
|
||||
---
|
||||
|
||||
### Requirement: 文件存在性检查
|
||||
|
||||
系统 SHALL 支持检查文件是否存在于对象存储。
|
||||
|
||||
#### Scenario: 文件存在
|
||||
- **WHEN** 调用 `Exists(ctx, "imports/test.csv")` 且文件存在
|
||||
- **THEN** 方法 SHALL 返回 (true, nil)
|
||||
|
||||
#### Scenario: 文件不存在
|
||||
- **WHEN** 调用 `Exists(ctx, "imports/test.csv")` 且文件不存在
|
||||
- **THEN** 方法 SHALL 返回 (false, nil)
|
||||
|
||||
---
|
||||
|
||||
### Requirement: 预签名上传 URL
|
||||
|
||||
系统 SHALL 支持生成预签名上传 URL,允许前端直接上传文件到对象存储。
|
||||
|
||||
#### Scenario: 生成上传 URL
|
||||
- **WHEN** 调用 `GetUploadURL(ctx, "imports/test.csv", "text/csv", 15*time.Minute)`
|
||||
- **THEN** 方法 SHALL 返回有效的预签名 URL
|
||||
- **THEN** URL SHALL 在指定时间(15分钟)后过期
|
||||
- **THEN** 使用该 URL 的 PUT 请求 SHALL 能成功上传文件
|
||||
|
||||
#### Scenario: URL 过期后
|
||||
- **WHEN** 使用过期的预签名 URL 上传
|
||||
- **THEN** 对象存储 SHALL 返回 403 Forbidden
|
||||
|
||||
---
|
||||
|
||||
### Requirement: 预签名下载 URL
|
||||
|
||||
系统 SHALL 支持生成预签名下载 URL,允许用户直接从对象存储下载文件。
|
||||
|
||||
#### Scenario: 生成下载 URL
|
||||
- **WHEN** 调用 `GetDownloadURL(ctx, "exports/report.xlsx", 24*time.Hour)`
|
||||
- **THEN** 方法 SHALL 返回有效的预签名 URL
|
||||
- **THEN** URL SHALL 在指定时间(24小时)后过期
|
||||
- **THEN** 使用该 URL 的 GET 请求 SHALL 能下载文件
|
||||
|
||||
---
|
||||
|
||||
### Requirement: 获取上传 URL API
|
||||
|
||||
系统 SHALL 提供 API 接口供前端获取预签名上传 URL。
|
||||
|
||||
接口定义:
|
||||
```
|
||||
POST /api/admin/storage/upload-url
|
||||
Authorization: Bearer <token>
|
||||
Content-Type: application/json
|
||||
|
||||
Request:
|
||||
{
|
||||
"file_name": "cards.csv",
|
||||
"content_type": "text/csv",
|
||||
"purpose": "iot_import"
|
||||
}
|
||||
|
||||
Response:
|
||||
{
|
||||
"code": 0,
|
||||
"message": "success",
|
||||
"data": {
|
||||
"upload_url": "http://obs-helf.cucloud.cn/cmp/imports/2025/01/24/abc123.csv?X-Amz-...",
|
||||
"file_key": "imports/2025/01/24/abc123.csv",
|
||||
"expires_in": 900
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
#### Scenario: 获取上传 URL 成功
|
||||
- **WHEN** 已认证用户调用 POST /api/admin/storage/upload-url
|
||||
- **AND** 请求包含有效的 file_name、content_type、purpose
|
||||
- **THEN** 系统 SHALL 返回预签名上传 URL 和 file_key
|
||||
- **THEN** file_key 格式 SHALL 为 `{purpose}/{year}/{month}/{day}/{uuid}.{ext}`
|
||||
|
||||
#### Scenario: 参数缺失
|
||||
- **WHEN** 请求缺少必填参数
|
||||
- **THEN** 系统 SHALL 返回 400 错误
|
||||
|
||||
#### Scenario: 未认证
|
||||
- **WHEN** 请求未携带有效 Token
|
||||
- **THEN** 系统 SHALL 返回 401 错误
|
||||
|
||||
---
|
||||
|
||||
### Requirement: 文件路径规范
|
||||
|
||||
系统 SHALL 按照规范生成文件路径。
|
||||
|
||||
路径格式:`{purpose}/{year}/{month}/{day}/{uuid}.{ext}`
|
||||
|
||||
支持的 purpose 值:
|
||||
- `iot_import` → `imports/`
|
||||
- `export` → `exports/`
|
||||
- `attachment` → `attachments/`
|
||||
|
||||
#### Scenario: 生成导入文件路径
|
||||
- **WHEN** purpose 为 "iot_import",file_name 为 "cards.csv"
|
||||
- **THEN** 生成的 file_key SHALL 匹配 `imports/\d{4}/\d{2}/\d{2}/[a-f0-9-]+\.csv`
|
||||
|
||||
#### Scenario: 未知 purpose
|
||||
- **WHEN** purpose 值不在支持列表中
|
||||
- **THEN** 系统 SHALL 返回错误 "不支持的文件用途"
|
||||
|
||||
---
|
||||
|
||||
### Requirement: 配置结构
|
||||
|
||||
系统 SHALL 支持通过配置文件配置对象存储参数。
|
||||
|
||||
```yaml
|
||||
storage:
|
||||
provider: "s3"
|
||||
s3:
|
||||
endpoint: "http://obs-helf.cucloud.cn"
|
||||
region: "cn-langfang-2"
|
||||
bucket: "cmp"
|
||||
access_key_id: "${OSS_ACCESS_KEY_ID}"
|
||||
secret_access_key: "${OSS_SECRET_ACCESS_KEY}"
|
||||
use_ssl: false
|
||||
path_style: true
|
||||
presign:
|
||||
upload_expires: "15m"
|
||||
download_expires: "24h"
|
||||
temp_dir: "/tmp/junhong-storage"
|
||||
```
|
||||
|
||||
#### Scenario: 环境变量替换
|
||||
- **WHEN** 配置值为 `${ENV_VAR}` 格式
|
||||
- **THEN** 系统 SHALL 从环境变量读取实际值
|
||||
|
||||
#### Scenario: 临时目录不存在
|
||||
- **WHEN** temp_dir 目录不存在
|
||||
- **THEN** 系统 SHALL 自动创建该目录
|
||||
219
openspec/changes/archive/2026-01-24-add-object-storage/tasks.md
Normal file
219
openspec/changes/archive/2026-01-24-add-object-storage/tasks.md
Normal file
@@ -0,0 +1,219 @@
|
||||
# 对象存储集成 - 任务清单
|
||||
|
||||
## 1. 基础设施
|
||||
|
||||
- [x] 1.1 在 `pkg/config/config.go` 添加 Storage 配置结构体
|
||||
- [x] 1.2 在 `configs/config.yaml` 添加 storage 配置块(含环境变量占位符)
|
||||
- [x] 1.3 创建 `pkg/storage/` 目录结构
|
||||
|
||||
## 2. Provider 实现
|
||||
|
||||
- [x] 2.1 创建 `pkg/storage/storage.go` - Provider 接口定义
|
||||
- [x] 2.2 创建 `pkg/storage/types.go` - 公共类型(PresignResult、Config)
|
||||
- [x] 2.3 创建 `pkg/storage/s3.go` - S3Provider 实现
|
||||
- [x] 2.3.1 实现 NewS3Provider 构造函数
|
||||
- [x] 2.3.2 实现 Upload 方法
|
||||
- [x] 2.3.3 实现 Download 方法
|
||||
- [x] 2.3.4 实现 DownloadToTemp 方法(含 cleanup 函数)
|
||||
- [x] 2.3.5 实现 Delete 方法
|
||||
- [x] 2.3.6 实现 Exists 方法
|
||||
- [x] 2.3.7 实现 GetUploadURL 方法
|
||||
- [x] 2.3.8 实现 GetDownloadURL 方法
|
||||
- [x] 2.4 创建 `pkg/storage/service.go` - StorageService 封装
|
||||
- [x] 2.4.1 实现 GenerateFileKey 方法(purpose + 日期 + UUID)
|
||||
- [x] 2.4.2 实现 GetUploadURL 方法(生成 key + 获取预签名)
|
||||
- [x] 2.4.3 实现 DownloadToTemp 方法(透传 + 日志)
|
||||
|
||||
## 3. Bootstrap 集成
|
||||
|
||||
- [x] 3.1 在 `internal/bootstrap/` 添加 storage 初始化逻辑
|
||||
- [x] 3.2 在 `cmd/api/main.go` 集成 StorageService(可选,配置缺失时跳过)
|
||||
- [x] 3.3 在 `cmd/worker/main.go` 集成 StorageService
|
||||
|
||||
## 4. API 接口
|
||||
|
||||
- [x] 4.1 创建 `internal/model/dto/storage.go` - 请求/响应 DTO
|
||||
- [x] 4.1.1 GetUploadURLRequest(file_name, content_type, purpose)
|
||||
- [x] 4.1.2 GetUploadURLResponse(upload_url, file_key, expires_in)
|
||||
- [x] 4.2 创建 `internal/handler/admin/storage.go` - StorageHandler
|
||||
- [x] 4.2.1 实现 GetUploadURL 方法
|
||||
- [x] 4.3 注册路由 POST /api/admin/storage/upload-url
|
||||
- [x] 4.3.1 在 RouteSpec.Description 添加前端使用流程说明(Markdown 格式)
|
||||
- [x] 4.4 更新 `cmd/api/docs.go` 和 `cmd/gendocs/main.go` 添加 StorageHandler
|
||||
|
||||
## 5. ICCID 导入改造
|
||||
|
||||
- [x] 5.1 修改 `internal/model/iot_card_import_task.go`
|
||||
- [x] 5.1.1 添加 StorageBucket 字段
|
||||
- [x] 5.1.2 添加 StorageKey 字段
|
||||
- [x] 5.2 创建数据库迁移文件添加新字段
|
||||
- [x] 5.3 修改 `internal/model/dto/iot_card_import.go`
|
||||
- [x] 5.3.1 将 CreateImportTaskRequest 从 multipart 改为 JSON
|
||||
- [x] 5.3.2 添加 FileKey 字段,移除 File 字段
|
||||
- [x] 5.4 修改 `internal/handler/admin/iot_card_import.go`
|
||||
- [x] 5.4.1 移除 c.FormFile() 逻辑
|
||||
- [x] 5.4.2 改为接收 JSON body 解析 file_key
|
||||
- [x] 5.4.3 保存 storage_bucket 和 storage_key 到任务记录
|
||||
- [x] 5.6 更新导入接口的 RouteSpec.Description
|
||||
- [x] 5.6.1 说明接口变更(BREAKING: multipart → JSON)
|
||||
- [x] 5.6.2 说明完整导入流程(先获取上传 URL → 上传文件 → 调用导入接口)
|
||||
- [x] 5.5 修改 `internal/task/iot_card_import.go`
|
||||
- [x] 5.5.1 从任务记录获取 storage_key
|
||||
- [x] 5.5.2 调用 StorageService.DownloadToTemp 下载文件
|
||||
- [x] 5.5.3 处理完成后调用 cleanup() 删除临时文件
|
||||
- [x] 5.5.4 保留原有 CSV 解析逻辑
|
||||
|
||||
## 6. 错误码
|
||||
|
||||
- [x] 6.1 在 `pkg/errors/codes.go` 添加存储相关错误码
|
||||
- [x] 6.1.1 ErrStorageUploadFailed
|
||||
- [x] 6.1.2 ErrStorageDownloadFailed
|
||||
- [x] 6.1.3 ErrStorageFileNotFound
|
||||
- [x] 6.1.4 ErrStorageInvalidPurpose
|
||||
|
||||
## 7. 测试
|
||||
|
||||
- [x] 7.1 创建 `scripts/test_storage.go` - 对象存储功能验证脚本
|
||||
- [x] 7.2 联通云后台验证文件上传成功
|
||||
- [x] 7.3 现有 Worker 测试通过
|
||||
|
||||
## 8. 文档
|
||||
|
||||
- [x] 8.1 创建 `docs/object-storage/使用指南.md` - 后端开发指南
|
||||
- [x] 8.1.1 StorageService 使用示例
|
||||
- [x] 8.1.2 配置说明
|
||||
- [x] 8.1.3 错误处理
|
||||
- [x] 8.2 创建 `docs/object-storage/前端接入指南.md` - 前端接入说明
|
||||
- [x] 8.2.1 文件上传完整流程(时序图)
|
||||
- [x] 8.2.2 获取预签名 URL 接口说明
|
||||
- [x] 8.2.3 使用预签名 URL 上传文件(含代码示例)
|
||||
- [x] 8.2.4 ICCID 导入接口变更说明(BREAKING CHANGE)
|
||||
- [x] 8.2.5 错误处理和重试策略
|
||||
- [x] 8.3 更新 README.md 添加对象存储功能说明
|
||||
|
||||
---
|
||||
|
||||
## 附录:前端接入指南内容大纲
|
||||
|
||||
### A. 文件上传流程(时序图)
|
||||
|
||||
```
|
||||
前端 后端 API 对象存储
|
||||
│ │ │
|
||||
│ 1. POST /storage/upload-url │
|
||||
│ {file_name, content_type, purpose} │
|
||||
│ ─────────────────────────► │
|
||||
│ │ │
|
||||
│ 2. 返回 {upload_url, file_key, expires_in} │
|
||||
│ ◄───────────────────────── │
|
||||
│ │ │
|
||||
│ 3. PUT upload_url (文件内容) │
|
||||
│ ─────────────────────────────────────────────────► │
|
||||
│ │ │
|
||||
│ 4. 上传成功 (200 OK) │
|
||||
│ ◄───────────────────────────────────────────────── │
|
||||
│ │ │
|
||||
│ 5. POST /iot-cards/import │
|
||||
│ {carrier_id, batch_no, file_key} │
|
||||
│ ─────────────────────────► │
|
||||
│ │ │
|
||||
│ 6. 返回任务创建成功 │
|
||||
│ ◄───────────────────────── │
|
||||
```
|
||||
|
||||
### B. 接口说明(RouteSpec.Description 内容参考)
|
||||
|
||||
#### 获取上传 URL 接口
|
||||
|
||||
```markdown
|
||||
## 文件上传流程
|
||||
|
||||
### 第一步:获取预签名 URL
|
||||
调用本接口获取上传 URL 和 file_key。
|
||||
|
||||
### 第二步:直接上传到对象存储
|
||||
使用返回的 `upload_url` 发起 PUT 请求上传文件:
|
||||
\`\`\`javascript
|
||||
const response = await fetch(upload_url, {
|
||||
method: 'PUT',
|
||||
headers: { 'Content-Type': content_type },
|
||||
body: file
|
||||
});
|
||||
\`\`\`
|
||||
|
||||
### 第三步:使用 file_key 调用业务接口
|
||||
上传成功后,使用 `file_key` 调用相关业务接口(如 ICCID 导入)。
|
||||
|
||||
### 注意事项
|
||||
- 预签名 URL 有效期 15 分钟,请及时使用
|
||||
- 上传失败时可重新获取 URL 重试
|
||||
- file_key 在上传成功后永久有效
|
||||
|
||||
### purpose 可选值
|
||||
| 值 | 说明 | 生成路径 |
|
||||
|---|------|---------|
|
||||
| iot_import | ICCID 导入 | imports/YYYY/MM/DD/uuid.csv |
|
||||
| export | 数据导出 | exports/YYYY/MM/DD/uuid.xlsx |
|
||||
| attachment | 附件上传 | attachments/YYYY/MM/DD/uuid.ext |
|
||||
```
|
||||
|
||||
#### ICCID 导入接口
|
||||
|
||||
```markdown
|
||||
## ⚠️ 接口变更说明(BREAKING CHANGE)
|
||||
|
||||
本接口已从 `multipart/form-data` 改为 `application/json`。
|
||||
|
||||
### 变更前
|
||||
\`\`\`
|
||||
POST /api/admin/iot-cards/import
|
||||
Content-Type: multipart/form-data
|
||||
carrier_id, batch_no, file (文件)
|
||||
\`\`\`
|
||||
|
||||
### 变更后
|
||||
\`\`\`
|
||||
POST /api/admin/iot-cards/import
|
||||
Content-Type: application/json
|
||||
{
|
||||
"carrier_id": 1,
|
||||
"batch_no": "BATCH-2025-01",
|
||||
"file_key": "imports/2025/01/24/abc123.csv"
|
||||
}
|
||||
\`\`\`
|
||||
|
||||
### 完整导入流程
|
||||
1. 调用 `POST /api/admin/storage/upload-url` 获取上传 URL
|
||||
2. 使用预签名 URL 上传 CSV 文件
|
||||
3. 使用返回的 `file_key` 调用本接口
|
||||
```
|
||||
|
||||
### C. 前端代码示例(TypeScript)
|
||||
|
||||
```typescript
|
||||
// 完整的文件上传流程
|
||||
async function uploadAndImport(file: File, carrierId: number, batchNo: string) {
|
||||
// 1. 获取预签名 URL
|
||||
const { data } = await api.post('/storage/upload-url', {
|
||||
file_name: file.name,
|
||||
content_type: file.type || 'text/csv',
|
||||
purpose: 'iot_import'
|
||||
});
|
||||
|
||||
const { upload_url, file_key } = data;
|
||||
|
||||
// 2. 上传文件到对象存储
|
||||
await fetch(upload_url, {
|
||||
method: 'PUT',
|
||||
headers: { 'Content-Type': file.type || 'text/csv' },
|
||||
body: file
|
||||
});
|
||||
|
||||
// 3. 调用导入接口
|
||||
return api.post('/iot-cards/import', {
|
||||
carrier_id: carrierId,
|
||||
batch_no: batchNo,
|
||||
file_key: file_key
|
||||
});
|
||||
}
|
||||
```
|
||||
@@ -0,0 +1,2 @@
|
||||
schema: spec-driven
|
||||
created: 2026-01-24
|
||||
@@ -0,0 +1,114 @@
|
||||
## Context
|
||||
|
||||
当前项目使用 `github.com/swaggest/openapi-go/openapi3` 库生成 OpenAPI 3.0.3 规范文档。路由注册通过 `RouteSpec` 结构体传递元数据,但目前只支持 `Summary` 字段作为接口的简短描述。
|
||||
|
||||
OpenAPI 规范的 Operation 对象包含两个描述字段:
|
||||
- `summary`: 简短摘要(通常一行)
|
||||
- `description`: 详细说明,**支持 CommonMark Markdown 语法**
|
||||
|
||||
swaggest 库的 `openapi3.Operation` 结构体已包含 `Description *string` 字段,只需在代码中设置即可。
|
||||
|
||||
## Goals / Non-Goals
|
||||
|
||||
**Goals:**
|
||||
- 在 `RouteSpec` 中新增 `Description` 字段
|
||||
- 支持在接口文档中添加 Markdown 格式的详细说明
|
||||
- 保持向后兼容,Description 为可选字段
|
||||
- 更新 API 文档规范
|
||||
|
||||
**Non-Goals:**
|
||||
- 不修改 DTO 字段的 description 标签处理逻辑
|
||||
- 不修改现有路由注册代码(新字段可选)
|
||||
- 不扩展其他 OpenAPI 字段(如 externalDocs、deprecated 等)
|
||||
|
||||
## Decisions
|
||||
|
||||
### 决策 1: Description 字段类型
|
||||
|
||||
**选择**: 使用 `string` 类型
|
||||
|
||||
**原因**:
|
||||
- 与 `Summary` 字段保持一致
|
||||
- 空字符串表示无描述,语义清晰
|
||||
- 避免指针类型带来的 nil 检查复杂度
|
||||
|
||||
**备选方案**:
|
||||
- `*string` 指针类型 - 增加使用复杂度,无实际收益
|
||||
|
||||
### 决策 2: 函数签名变更策略
|
||||
|
||||
**选择**: 不修改 `AddOperation` 函数签名,通过 `RouteSpec.Description` 传递
|
||||
|
||||
**原因**:
|
||||
- 保持 API 稳定性
|
||||
- `Register` 函数已封装了 `RouteSpec`,只需从中提取 Description
|
||||
- 避免破坏性变更
|
||||
|
||||
**备选方案**:
|
||||
- 修改 `AddOperation` 增加 description 参数 - 需要修改所有调用点,不必要
|
||||
|
||||
### 决策 3: 空值处理
|
||||
|
||||
**选择**: 空字符串时不设置 OpenAPI 的 description 字段
|
||||
|
||||
**原因**:
|
||||
- 生成更简洁的 YAML
|
||||
- 与 swaggest 库的 omitempty 行为一致
|
||||
- 保持现有生成文件格式不变
|
||||
|
||||
## Risks / Trade-offs
|
||||
|
||||
**[风险] Markdown 语法在不同工具中渲染差异**
|
||||
→ 缓解: 建议使用 CommonMark 基础语法(标题、列表、表格、代码块),避免扩展语法
|
||||
|
||||
**[风险] 过长的 Description 影响文档可读性**
|
||||
→ 缓解: 在规范文档中建议控制长度,复杂说明可使用折叠或链接到外部文档
|
||||
|
||||
**[权衡] 不修改函数签名 vs 显式参数**
|
||||
→ 选择封装在 RouteSpec 中,牺牲一定的显式性换取稳定性
|
||||
|
||||
## 实现方案
|
||||
|
||||
### 文件变更清单
|
||||
|
||||
1. **`internal/routes/registry.go`**
|
||||
- RouteSpec 新增 Description 字段
|
||||
|
||||
2. **`pkg/openapi/generator.go`**
|
||||
- AddOperation: 设置 op.Description
|
||||
- AddMultipartOperation: 设置 op.Description
|
||||
|
||||
3. **`docs/api-documentation-guide.md`**
|
||||
- 新增 Description 字段使用说明
|
||||
- 补充 Markdown 语法示例
|
||||
|
||||
### 代码变更示例
|
||||
|
||||
```go
|
||||
// internal/routes/registry.go
|
||||
type RouteSpec struct {
|
||||
Summary string // 简短摘要
|
||||
Description string // 详细说明,支持 Markdown
|
||||
Input interface{}
|
||||
Output interface{}
|
||||
Tags []string
|
||||
Auth bool
|
||||
FileUploads []FileUploadField
|
||||
}
|
||||
```
|
||||
|
||||
```go
|
||||
// pkg/openapi/generator.go - AddOperation
|
||||
func (g *Generator) AddOperation(...) {
|
||||
op := openapi3.Operation{
|
||||
Summary: &summary,
|
||||
Tags: tags,
|
||||
}
|
||||
|
||||
// 新增: 设置 Description
|
||||
if description != "" {
|
||||
op.Description = &description
|
||||
}
|
||||
// ...
|
||||
}
|
||||
```
|
||||
@@ -0,0 +1,36 @@
|
||||
## Why
|
||||
|
||||
当前项目的 OpenAPI 文档生成模块只支持通过 `RouteSpec.Summary` 设置接口的简短摘要,无法添加详细的 Markdown 格式说明。前端团队使用 Apifox 查看 API 文档时,需要在某些接口上看到更详细的使用说明、注意事项、业务规则等信息。
|
||||
|
||||
OpenAPI 规范的 Operation 对象支持 `description` 字段,且该字段明确支持 **CommonMark** Markdown 语法。Apifox 作为 OpenAPI 工具,能够正确渲染这些 Markdown 内容。因此需要扩展当前的文档生成模块以支持此功能。
|
||||
|
||||
## What Changes
|
||||
|
||||
- 在 `RouteSpec` 结构体中新增 `Description` 字段,用于设置接口的详细 Markdown 说明
|
||||
- 修改 `pkg/openapi/generator.go` 中的 `AddOperation` 和 `AddMultipartOperation` 方法,将 Description 写入 OpenAPI 规范
|
||||
- 更新 `internal/routes/registry.go` 中的 `Register` 函数以传递 Description 参数
|
||||
- 更新 API 文档规范,说明 Description 字段的使用方法和 Markdown 语法支持
|
||||
|
||||
## Capabilities
|
||||
|
||||
### New Capabilities
|
||||
|
||||
- `openapi-markdown-description`: 支持在 OpenAPI 接口文档中添加 Markdown 格式的详细描述,包括表格、列表、代码块等富文本内容
|
||||
|
||||
### Modified Capabilities
|
||||
|
||||
<!-- 无现有能力需要修改,这是纯新增功能 -->
|
||||
|
||||
## Impact
|
||||
|
||||
**受影响的代码**:
|
||||
- `internal/routes/registry.go` - RouteSpec 结构体定义和 Register 函数
|
||||
- `pkg/openapi/generator.go` - AddOperation 和 AddMultipartOperation 方法
|
||||
|
||||
**受影响的文档**:
|
||||
- `docs/api-documentation-guide.md` - 需要补充 Description 字段使用说明
|
||||
|
||||
**不受影响**:
|
||||
- 所有现有路由注册代码(Description 字段为可选,空值时行为与当前一致)
|
||||
- 生成的 OpenAPI 文件格式(符合 OpenAPI 3.0.3 规范)
|
||||
- Apifox 导入流程(无需任何配置变更)
|
||||
@@ -0,0 +1,56 @@
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: RouteSpec 支持 Description 字段
|
||||
|
||||
RouteSpec 结构体 SHALL 包含 `Description` 字段,类型为 `string`,用于设置接口的详细 Markdown 说明。
|
||||
|
||||
#### Scenario: Description 字段为空时不影响生成
|
||||
|
||||
- **WHEN** RouteSpec.Description 为空字符串
|
||||
- **THEN** 生成的 OpenAPI 规范中该接口不包含 description 字段
|
||||
|
||||
#### Scenario: Description 字段有内容时写入 OpenAPI
|
||||
|
||||
- **WHEN** RouteSpec.Description 包含非空内容
|
||||
- **THEN** 生成的 OpenAPI 规范中该接口的 description 字段包含该内容
|
||||
|
||||
### Requirement: Description 支持 Markdown 语法
|
||||
|
||||
生成器 SHALL 原样保留 Description 字段的 Markdown 内容,不进行转义或处理,以便 OpenAPI 工具(如 Apifox)正确渲染。
|
||||
|
||||
#### Scenario: 支持基础 Markdown 格式
|
||||
|
||||
- **WHEN** Description 包含 Markdown 标题、列表、表格、代码块
|
||||
- **THEN** 生成的 OpenAPI YAML 文件中保留完整的 Markdown 格式
|
||||
|
||||
#### Scenario: 支持多行内容
|
||||
|
||||
- **WHEN** Description 包含多行文本
|
||||
- **THEN** 生成的 OpenAPI YAML 文件使用 YAML 多行字符串格式正确表示
|
||||
|
||||
### Requirement: AddOperation 方法处理 Description
|
||||
|
||||
AddOperation 方法 SHALL 接受 description 参数并设置到 openapi3.Operation.Description 字段。
|
||||
|
||||
#### Scenario: 普通接口设置 Description
|
||||
|
||||
- **WHEN** 调用 AddOperation 且 description 参数非空
|
||||
- **THEN** 生成的 Operation 对象包含 Description 字段
|
||||
|
||||
### Requirement: AddMultipartOperation 方法处理 Description
|
||||
|
||||
AddMultipartOperation 方法 SHALL 与 AddOperation 一致,支持 description 参数。
|
||||
|
||||
#### Scenario: 文件上传接口设置 Description
|
||||
|
||||
- **WHEN** 调用 AddMultipartOperation 且 description 参数非空
|
||||
- **THEN** 生成的 multipart/form-data 接口包含 Description 字段
|
||||
|
||||
### Requirement: Register 函数传递 Description
|
||||
|
||||
Register 函数 SHALL 从 RouteSpec 中提取 Description 字段并传递给文档生成器。
|
||||
|
||||
#### Scenario: Register 调用时传递 Description
|
||||
|
||||
- **WHEN** 调用 Register 函数注册路由
|
||||
- **THEN** RouteSpec.Description 被传递到对应的 AddOperation 或 AddMultipartOperation 调用
|
||||
@@ -0,0 +1,24 @@
|
||||
## 1. RouteSpec 结构体扩展
|
||||
|
||||
- [x] 1.1 在 `internal/routes/registry.go` 的 RouteSpec 结构体中新增 Description 字段
|
||||
|
||||
## 2. OpenAPI 生成器修改
|
||||
|
||||
- [x] 2.1 修改 `pkg/openapi/generator.go` 的 AddOperation 方法签名,增加 description 参数
|
||||
- [x] 2.2 在 AddOperation 方法中设置 op.Description 字段
|
||||
- [x] 2.3 修改 AddMultipartOperation 方法签名,增加 description 参数
|
||||
- [x] 2.4 在 AddMultipartOperation 方法中设置 op.Description 字段
|
||||
|
||||
## 3. Register 函数更新
|
||||
|
||||
- [x] 3.1 更新 `internal/routes/registry.go` 的 Register 函数,传递 spec.Description 给生成器
|
||||
|
||||
## 4. 文档更新
|
||||
|
||||
- [x] 4.1 更新 `docs/api-documentation-guide.md`,新增 Description 字段使用说明
|
||||
- [x] 4.2 补充 Markdown 语法示例和最佳实践
|
||||
|
||||
## 5. 验证
|
||||
|
||||
- [x] 5.1 运行 `go run cmd/gendocs/main.go` 验证文档生成正常
|
||||
- [x] 5.2 检查生成的 YAML 文件格式正确
|
||||
@@ -0,0 +1,2 @@
|
||||
schema: spec-driven
|
||||
created: 2026-01-24
|
||||
@@ -0,0 +1,35 @@
|
||||
# Change: 修复 ICCID 导入 CSV 格式支持 MSISDN
|
||||
|
||||
## Why
|
||||
|
||||
当前批量导入 ICCID 接口只支持单列 CSV(仅 ICCID),但 IoT 卡模型包含 MSISDN(接入号/手机号码)字段,导入时无法填充该重要字段。运营商提供的卡资料必须同时包含 ICCID 和 MSISDN,缺少接入号的卡无法正常使用。
|
||||
|
||||
## What Changes
|
||||
|
||||
- **BREAKING**: CSV 格式变更为必须包含两列(ICCID, MSISDN),不再支持单列格式
|
||||
- 修改 CSV 解析逻辑要求两列格式,缺少 MSISDN 的行视为格式错误
|
||||
- 修改导入任务模型存储 ICCID 和 MSISDN 的映射关系
|
||||
- 修改导入任务处理逻辑,创建卡记录时填充 MSISDN 字段
|
||||
- 更新 API 文档描述新的 CSV 格式要求
|
||||
|
||||
## Capabilities
|
||||
|
||||
### New Capabilities
|
||||
|
||||
(无新增能力)
|
||||
|
||||
### Modified Capabilities
|
||||
|
||||
- `iot-card-import-task`: 导入任务需要存储 ICCID-MSISDN 映射,CSV 解析结果结构变更,强制要求双列格式
|
||||
|
||||
## Impact
|
||||
|
||||
- 受影响的代码:
|
||||
- `pkg/utils/csv.go` - CSV 解析函数
|
||||
- `internal/model/iot_card_import_task.go` - 导入任务模型
|
||||
- `internal/task/iot_card_import.go` - 导入任务处理逻辑
|
||||
- `internal/service/iot_card_import/service.go` - 导入服务
|
||||
- 受影响的 API:
|
||||
- `POST /api/admin/iot-cards/import` - CSV 格式要求变更(**BREAKING**)
|
||||
- 数据库: 需要迁移更新 `iccid_list` 字段结构(从 `[]string` 改为 `[{iccid, msisdn}]`)
|
||||
- 用户影响: 现有单列 CSV 文件需要补充 MSISDN 列后才能导入
|
||||
@@ -0,0 +1,126 @@
|
||||
## MODIFIED Requirements
|
||||
|
||||
### Requirement: 导入任务实体定义
|
||||
|
||||
系统 SHALL 定义 IoT 卡导入任务(IotCardImportTask)实体,用于跟踪 IoT 卡批量导入的进度和结果。
|
||||
|
||||
**实体字段**:
|
||||
|
||||
**任务信息**:
|
||||
- `id`: 任务 ID(主键,BIGINT)
|
||||
- `task_no`: 任务编号(VARCHAR(50),唯一,格式: IMP-YYYYMMDD-XXXXXX)
|
||||
- `status`: 任务状态(INT,1-待处理 2-处理中 3-已完成 4-失败)
|
||||
|
||||
**导入参数**:
|
||||
- `carrier_id`: 运营商 ID(BIGINT,必填)
|
||||
- `carrier_type`: 运营商类型(VARCHAR(10),CMCC/CUCC/CTCC/CBN)
|
||||
- `batch_no`: 批次号(VARCHAR(100),可选)
|
||||
- `file_name`: 原始文件名(VARCHAR(255),可选)
|
||||
|
||||
**待导入数据**:
|
||||
- `card_list`: 待导入卡列表(JSONB,结构: [{iccid, msisdn}],替代原 iccid_list)
|
||||
|
||||
**进度统计**:
|
||||
- `total_count`: 总数(INT,CSV 文件总行数)
|
||||
- `success_count`: 成功数(INT,成功导入的卡数量)
|
||||
- `skip_count`: 跳过数(INT,因重复等原因跳过的数量)
|
||||
- `fail_count`: 失败数(INT,因格式错误等原因失败的数量)
|
||||
|
||||
**结果详情**:
|
||||
- `skipped_items`: 跳过记录详情(JSONB,结构: [{line, iccid, msisdn, reason}])
|
||||
- `failed_items`: 失败记录详情(JSONB,结构: [{line, iccid, msisdn, reason}])
|
||||
|
||||
**时间和错误**:
|
||||
- `started_at`: 开始处理时间(TIMESTAMP,可空)
|
||||
- `completed_at`: 完成时间(TIMESTAMP,可空)
|
||||
- `error_message`: 任务级错误信息(TEXT,可空,如文件解析失败等)
|
||||
|
||||
**系统字段**:
|
||||
- `shop_id`: 店铺 ID(BIGINT,可空,记录发起导入的店铺)
|
||||
- `created_at`: 创建时间(TIMESTAMP,自动填充)
|
||||
- `updated_at`: 更新时间(TIMESTAMP,自动填充)
|
||||
- `creator`: 创建人 ID(BIGINT)
|
||||
- `updater`: 更新人 ID(BIGINT)
|
||||
|
||||
#### Scenario: 创建导入任务
|
||||
|
||||
- **GIVEN** 管理员上传包含 ICCID 和 MSISDN 两列的 CSV 文件
|
||||
- **WHEN** 系统解析 CSV 并创建导入任务
|
||||
- **THEN** 系统创建导入任务记录,`card_list` 包含 [{iccid, msisdn}] 结构,`status` 为 1(待处理)
|
||||
|
||||
---
|
||||
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: CSV 文件格式规范
|
||||
|
||||
系统 SHALL 要求 CSV 文件必须包含 ICCID 和 MSISDN 两列。
|
||||
|
||||
**文件格式要求**:
|
||||
- 第一列: ICCID(必填,不能为空)
|
||||
- 第二列: MSISDN/接入号(必填,不能为空)
|
||||
- 支持表头行(自动识别并跳过)
|
||||
- 表头识别关键字: iccid/卡号 + msisdn/接入号/手机号
|
||||
|
||||
**解析规则**:
|
||||
- 自动去除首尾空格
|
||||
- 跳过空行
|
||||
- 第一行为表头时自动跳过
|
||||
- 列数不足 2 列的文件拒绝导入
|
||||
- ICCID 为空的行记录为失败
|
||||
- MSISDN 为空的行记录为失败
|
||||
|
||||
#### Scenario: 解析标准双列 CSV 文件
|
||||
|
||||
- **GIVEN** CSV 文件内容为:
|
||||
```
|
||||
iccid,msisdn
|
||||
89860012345678901234,13800000001
|
||||
89860012345678901235,13800000002
|
||||
```
|
||||
- **WHEN** 系统解析该 CSV 文件
|
||||
- **THEN** 解析结果包含 2 条有效记录,每条包含 ICCID 和 MSISDN
|
||||
|
||||
#### Scenario: 拒绝单列 CSV 文件
|
||||
|
||||
- **GIVEN** CSV 文件内容仅包含 ICCID 单列
|
||||
- **WHEN** 系统尝试解析该 CSV 文件
|
||||
- **THEN** 系统返回错误 "CSV 文件格式错误:缺少 MSISDN 列"
|
||||
|
||||
#### Scenario: MSISDN 为空的行记录失败
|
||||
|
||||
- **GIVEN** CSV 文件内容为:
|
||||
```
|
||||
iccid,msisdn
|
||||
89860012345678901234,13800000001
|
||||
89860012345678901235,
|
||||
```
|
||||
- **WHEN** 系统解析该 CSV 文件
|
||||
- **THEN** 第一条记录解析成功,第二条记录标记为失败,原因为 "MSISDN 不能为空"
|
||||
|
||||
#### Scenario: ICCID 为空的行记录失败
|
||||
|
||||
- **GIVEN** CSV 文件内容为:
|
||||
```
|
||||
iccid,msisdn
|
||||
89860012345678901234,13800000001
|
||||
,13800000002
|
||||
```
|
||||
- **WHEN** 系统解析该 CSV 文件
|
||||
- **THEN** 第一条记录解析成功,第二条记录标记为失败,原因为 "ICCID 不能为空"
|
||||
|
||||
---
|
||||
|
||||
### Requirement: 导入时填充 MSISDN 字段
|
||||
|
||||
系统 SHALL 在创建 IoT 卡记录时填充 MSISDN 字段。
|
||||
|
||||
**处理规则**:
|
||||
- 从 `card_list` 中获取 ICCID 和 MSISDN
|
||||
- 创建 `IotCard` 记录时同时设置 `iccid` 和 `msisdn` 字段
|
||||
|
||||
#### Scenario: 创建卡记录时填充 MSISDN
|
||||
|
||||
- **GIVEN** 导入任务包含卡数据 [{iccid: "898600...", msisdn: "13800000001"}]
|
||||
- **WHEN** Worker 处理导入任务创建卡记录
|
||||
- **THEN** 创建的 `IotCard` 记录 `iccid` 为 "898600...",`msisdn` 为 "13800000001"
|
||||
@@ -0,0 +1,31 @@
|
||||
## 1. CSV 解析改造
|
||||
|
||||
- [x] 1.1 修改 `pkg/utils/csv.go` 的 `CSVParseResult` 结构体,添加 `Cards []CardInfo` 替代 `ICCIDs []string`
|
||||
- [x] 1.2 修改 `ParseICCIDFromCSV` 函数为 `ParseCardCSV`,支持解析 ICCID + MSISDN 两列
|
||||
- [x] 1.3 添加列数校验,单列 CSV 直接返回错误
|
||||
- [x] 1.4 添加 ICCID/MSISDN 非空校验,空值记录为解析错误
|
||||
- [x] 1.5 更新表头识别逻辑,支持 msisdn/接入号/手机号 关键字
|
||||
- [x] 1.6 更新 `pkg/utils/csv_test.go` 测试用例
|
||||
|
||||
## 2. 导入任务模型改造
|
||||
|
||||
- [x] 2.1 创建数据库迁移:将 `iccid_list` 字段重命名为 `card_list`,类型保持 JSONB
|
||||
- [x] 2.2 修改 `internal/model/iot_card_import_task.go`,定义 `CardListJSON` 类型为 `[]CardInfo{ICCID, MSISDN}`
|
||||
- [x] 2.3 更新 `ImportResultItem` 结构体添加 `MSISDN` 字段
|
||||
|
||||
## 3. 导入服务改造
|
||||
|
||||
- [x] 3.1 修改 `internal/service/iot_card_import/service.go`,调用新的 `ParseCardCSV` 函数
|
||||
- [x] 3.2 将解析结果的 `Cards` 存入任务的 `card_list` 字段
|
||||
|
||||
## 4. 导入任务处理改造
|
||||
|
||||
- [x] 4.1 修改 `internal/task/iot_card_import.go` 的 `getICCIDsFromTask` 改为 `getCardsFromTask`
|
||||
- [x] 4.2 修改 `processBatch` 函数,创建卡记录时同时填充 `ICCID` 和 `MSISDN`
|
||||
- [x] 4.3 更新失败/跳过记录的结构,包含 MSISDN 信息
|
||||
- [x] 4.4 更新 `internal/task/iot_card_import_test.go` 测试用例
|
||||
|
||||
## 5. API 文档更新
|
||||
|
||||
- [x] 5.1 更新路由注册中 CSV 文件字段的描述,说明必须包含 ICCID 和 MSISDN 两列
|
||||
- [x] 5.2 重新生成 OpenAPI 文档
|
||||
@@ -0,0 +1,2 @@
|
||||
schema: spec-driven
|
||||
created: 2026-01-24
|
||||
@@ -0,0 +1,281 @@
|
||||
# 技术设计:部署自初始化
|
||||
|
||||
## Context
|
||||
|
||||
### 当前状态
|
||||
|
||||
1. **目录创建分散**
|
||||
- `pkg/storage/s3.go` 在初始化时创建临时目录
|
||||
- 日志目录依赖 Dockerfile 预创建
|
||||
- 没有统一的初始化入口
|
||||
|
||||
2. **配置管理复杂**
|
||||
- 4 个外部配置文件:`config.yaml`、`config.dev.yaml`、`config.staging.yaml`、`config.prod.yaml`
|
||||
- `pkg/config/watcher.go` 实现热重载(开发阶段不需要)
|
||||
- 必须手动拷贝配置文件才能启动
|
||||
|
||||
3. **部署流程繁琐**
|
||||
- 需要手动创建目录结构
|
||||
- 需要手动拷贝配置文件
|
||||
- Docker Compose 挂载 configs 目录
|
||||
|
||||
### 约束
|
||||
|
||||
- 使用 Go 1.16+ 的 `go:embed` 特性
|
||||
- 保持 Viper 作为配置解析库
|
||||
- 容器内以非 root 用户 (appuser:1000) 运行
|
||||
|
||||
## Goals / Non-Goals
|
||||
|
||||
**Goals:**
|
||||
- 应用启动时自动创建所有必需目录
|
||||
- 配置嵌入二进制,无需外部配置文件
|
||||
- 环境变量作为配置覆盖机制
|
||||
- 部署流程简化到 1 步
|
||||
|
||||
**Non-Goals:**
|
||||
- 配置热重载(开发阶段移除)
|
||||
- 多配置文件支持(统一用嵌入默认值 + 环境变量)
|
||||
- 向后兼容旧的配置方式
|
||||
|
||||
## Decisions
|
||||
|
||||
### Decision 1: 目录初始化放在应用层
|
||||
|
||||
**选择**: 在 `main.go` 启动时调用集中化的目录初始化函数
|
||||
|
||||
**备选方案**:
|
||||
| 方案 | 优点 | 缺点 |
|
||||
|------|------|------|
|
||||
| Dockerfile RUN mkdir | 镜像固定 | 不支持运行时配置路径 |
|
||||
| Entrypoint 脚本 | 运行时动态 | Shell 脚本维护成本高 |
|
||||
| **应用代码初始化** | 跨环境通用、错误处理清晰 | 无 |
|
||||
|
||||
**理由**: 应用代码可以读取配置中的路径,提供 Go 级别的错误处理,并在所有部署环境(Docker、K8s、裸机)通用。
|
||||
|
||||
### Decision 2: 配置嵌入 + 环境变量覆盖
|
||||
|
||||
**选择**: 使用 `go:embed` 嵌入默认配置,环境变量覆盖
|
||||
|
||||
**配置优先级**:
|
||||
```
|
||||
环境变量 (JUNHONG_*) > 嵌入默认值
|
||||
```
|
||||
|
||||
**备选方案**:
|
||||
| 方案 | 优点 | 缺点 |
|
||||
|------|------|------|
|
||||
| 纯外部文件 | 运行时可改 | 部署复杂 |
|
||||
| 纯环境变量 | 12-Factor | 复杂配置难表达 |
|
||||
| **嵌入 + 环境变量** | 开箱即用 + 灵活覆盖 | 改默认值需重编译 |
|
||||
|
||||
**理由**: 嵌入配置保证"开箱即用",环境变量覆盖满足生产环境定制需求。开发阶段不需要频繁改默认值。
|
||||
|
||||
### Decision 3: 环境变量命名规范
|
||||
|
||||
**选择**: `JUNHONG_` 前缀 + 下划线分隔
|
||||
|
||||
**格式**: `JUNHONG_{SECTION}_{KEY}`
|
||||
|
||||
**示例**:
|
||||
```bash
|
||||
JUNHONG_DATABASE_HOST=localhost
|
||||
JUNHONG_DATABASE_PORT=5432
|
||||
JUNHONG_REDIS_ADDRESS=localhost:6379
|
||||
JUNHONG_SERVER_ADDRESS=:3000
|
||||
```
|
||||
|
||||
**理由**:
|
||||
- 前缀避免与系统环境变量冲突
|
||||
- 下划线分隔便于 Viper 的 `SetEnvKeyReplacer` 处理
|
||||
- 符合业界惯例(类似 `POSTGRES_`、`REDIS_` 等)
|
||||
|
||||
### Decision 4: 删除配置热重载
|
||||
|
||||
**选择**: 删除 `pkg/config/watcher.go` 及相关逻辑
|
||||
|
||||
**理由**:
|
||||
- 开发阶段,配置变更频率低
|
||||
- 重启容器即可应用新配置
|
||||
- 减少代码复杂度
|
||||
|
||||
### Decision 5: 目录降级策略
|
||||
|
||||
**选择**: 权限不足时使用系统临时目录作为 fallback
|
||||
|
||||
```go
|
||||
if err := os.MkdirAll(dir, 0755); err != nil {
|
||||
if os.IsPermission(err) {
|
||||
fallback := filepath.Join(os.TempDir(), "junhong", filepath.Base(dir))
|
||||
os.MkdirAll(fallback, 0755)
|
||||
return fallback, nil
|
||||
}
|
||||
return "", err
|
||||
}
|
||||
```
|
||||
|
||||
**理由**: 提高容错性,即使权限配置不当也能启动(降级运行)。
|
||||
|
||||
## Architecture
|
||||
|
||||
### 启动流程
|
||||
|
||||
```
|
||||
main.go
|
||||
│
|
||||
├── 1. config.Load() // 加载嵌入配置 + 环境变量覆盖
|
||||
│ ├── 读取 go:embed 的 defaults/config.yaml
|
||||
│ ├── 应用 JUNHONG_* 环境变量覆盖
|
||||
│ └── 验证必填配置
|
||||
│
|
||||
├── 2. bootstrap.EnsureDirectories(cfg) // 创建所有必需目录
|
||||
│ ├── 临时文件目录
|
||||
│ ├── 日志目录
|
||||
│ └── 其他运行时目录
|
||||
│
|
||||
├── 3. logger.Init(cfg) // 初始化日志
|
||||
│
|
||||
├── 4. database.Init(cfg) // 初始化数据库
|
||||
│
|
||||
└── 5. ... 其他组件初始化
|
||||
```
|
||||
|
||||
### 目录结构
|
||||
|
||||
```
|
||||
pkg/
|
||||
├── bootstrap/
|
||||
│ └── directories.go # 新增:目录初始化
|
||||
├── config/
|
||||
│ ├── config.go # 保留:配置结构定义
|
||||
│ ├── loader.go # 重写:嵌入配置加载
|
||||
│ ├── embedded.go # 新增:go:embed 逻辑
|
||||
│ ├── defaults/
|
||||
│ │ └── config.yaml # 新增:嵌入的默认配置
|
||||
│ └── watcher.go # 删除
|
||||
```
|
||||
|
||||
### 嵌入配置内容
|
||||
|
||||
```yaml
|
||||
# pkg/config/defaults/config.yaml
|
||||
server:
|
||||
address: ":3000"
|
||||
read_timeout: 30s
|
||||
write_timeout: 30s
|
||||
shutdown_timeout: 30s
|
||||
prefork: false
|
||||
|
||||
database:
|
||||
host: "" # 必须通过 JUNHONG_DATABASE_HOST 设置
|
||||
port: 5432
|
||||
user: "" # 必须通过 JUNHONG_DATABASE_USER 设置
|
||||
password: "" # 必须通过 JUNHONG_DATABASE_PASSWORD 设置
|
||||
dbname: "" # 必须通过 JUNHONG_DATABASE_DBNAME 设置
|
||||
sslmode: "disable"
|
||||
max_open_conns: 25
|
||||
max_idle_conns: 10
|
||||
conn_max_lifetime: 5m
|
||||
|
||||
redis:
|
||||
address: "" # 必须通过 JUNHONG_REDIS_ADDRESS 设置
|
||||
port: 6379
|
||||
password: ""
|
||||
db: 0
|
||||
pool_size: 10
|
||||
min_idle_conns: 5
|
||||
dial_timeout: 5s
|
||||
read_timeout: 3s
|
||||
write_timeout: 3s
|
||||
|
||||
storage:
|
||||
provider: "s3"
|
||||
temp_dir: "/tmp/junhong-storage"
|
||||
s3:
|
||||
endpoint: ""
|
||||
region: ""
|
||||
bucket: ""
|
||||
access_key_id: ""
|
||||
secret_access_key: ""
|
||||
use_ssl: false
|
||||
path_style: true
|
||||
presign:
|
||||
upload_expires: 15m
|
||||
download_expires: 24h
|
||||
|
||||
logging:
|
||||
level: "info"
|
||||
development: false
|
||||
app_log:
|
||||
filename: "/app/logs/app.log"
|
||||
max_size: 100
|
||||
max_backups: 3
|
||||
max_age: 7
|
||||
compress: true
|
||||
access_log:
|
||||
filename: "/app/logs/access.log"
|
||||
max_size: 100
|
||||
max_backups: 3
|
||||
max_age: 7
|
||||
compress: true
|
||||
|
||||
queue:
|
||||
concurrency: 10
|
||||
retry_max: 5
|
||||
timeout: 10m
|
||||
|
||||
jwt:
|
||||
secret_key: "" # 必须通过 JUNHONG_JWT_SECRET_KEY 设置
|
||||
token_duration: 24h
|
||||
access_token_ttl: 24h
|
||||
refresh_token_ttl: 168h
|
||||
|
||||
middleware:
|
||||
enable_rate_limiter: false
|
||||
rate_limiter:
|
||||
max: 100
|
||||
expiration: 1m
|
||||
storage: "memory"
|
||||
```
|
||||
|
||||
## Risks / Trade-offs
|
||||
|
||||
| 风险 | 影响 | 缓解措施 |
|
||||
|------|------|----------|
|
||||
| 嵌入配置修改需重新编译 | 低(开发阶段) | 敏感配置通过环境变量,嵌入的只是默认值 |
|
||||
| 环境变量泄露 | 中 | 使用 K8s Secrets 或 Docker Secrets 管理 |
|
||||
| 目录降级后行为不一致 | 低 | 降级时打印 WARN 日志,明确告知 |
|
||||
| 删除热重载后调试不便 | 低 | 开发时直接重启进程,生产用 rolling update |
|
||||
|
||||
## Migration Plan
|
||||
|
||||
### 实施步骤
|
||||
|
||||
1. **新增目录初始化模块**
|
||||
- 创建 `pkg/bootstrap/directories.go`
|
||||
- 在 main.go 中调用
|
||||
|
||||
2. **新增配置嵌入模块**
|
||||
- 创建 `pkg/config/defaults/config.yaml`
|
||||
- 创建 `pkg/config/embedded.go`
|
||||
- 重写 `pkg/config/loader.go`
|
||||
|
||||
3. **清理旧配置逻辑**
|
||||
- 删除 `pkg/config/watcher.go`
|
||||
- 删除 `configs/*.yaml`
|
||||
|
||||
4. **更新 Docker 配置**
|
||||
- 更新 `Dockerfile.api` 和 `Dockerfile.worker`
|
||||
- 重写 `docker-compose.prod.yml`
|
||||
|
||||
5. **更新文档**
|
||||
- 更新 README.md 部署说明
|
||||
- 更新环境变量文档
|
||||
|
||||
### 回滚策略
|
||||
|
||||
如需回滚,恢复 git 提交即可(开发阶段无生产数据风险)。
|
||||
|
||||
## Open Questions
|
||||
|
||||
无。设计已明确,可直接实施。
|
||||
@@ -0,0 +1,104 @@
|
||||
# 提案:部署自初始化
|
||||
|
||||
## Why
|
||||
|
||||
当前应用部署需要手动创建目录结构、拷贝配置文件,过程繁琐且容易出错。临时目录创建逻辑分散在各组件中,非 root 用户可能因权限问题导致启动失败。配置文件必须外部提供,无法实现"开箱即用"的部署体验。
|
||||
|
||||
本变更旨在实现**应用自初始化**,让应用启动时自动创建所需目录、使用嵌入的默认配置,大幅简化部署流程。
|
||||
|
||||
## What Changes
|
||||
|
||||
### 1. 集中化目录初始化
|
||||
- 新增 `pkg/bootstrap/directories.go`,在应用启动时统一创建所有必需目录
|
||||
- 移除各组件(如 `s3.go`)中分散的目录创建逻辑
|
||||
- 提供降级策略:权限不足时自动使用备用路径
|
||||
|
||||
### 2. 配置嵌入机制
|
||||
- 使用 `go:embed` 将默认配置嵌入二进制文件
|
||||
- 配置优先级:**环境变量 > 嵌入默认值**(移除外部配置文件依赖)
|
||||
- 敏感配置(数据库密码等)通过环境变量提供
|
||||
- **移除** configs/ 目录和配置文件热重载机制(开发阶段不需要)
|
||||
|
||||
### 3. Docker 部署简化
|
||||
- Dockerfile 预创建关键目录并设置正确权限
|
||||
- **移除** configs 目录挂载,全部使用环境变量
|
||||
- docker-compose 只挂载日志目录(持久化需求)
|
||||
|
||||
### 4. 环境变量规范化
|
||||
- 统一环境变量前缀:`JUNHONG_`
|
||||
- 支持嵌套配置:`JUNHONG_DATABASE_HOST`、`JUNHONG_REDIS_ADDRESS`
|
||||
- 配置验证:必填配置未设置时启动失败并给出明确提示
|
||||
|
||||
### 5. 清理冗余
|
||||
- **删除** `pkg/config/watcher.go`(配置热重载)
|
||||
- **删除** `configs/*.yaml` 外部配置文件
|
||||
- **删除** docker-compose 中的 configs 卷挂载
|
||||
- **简化** `pkg/config/loader.go`
|
||||
|
||||
## Capabilities
|
||||
|
||||
### New Capabilities
|
||||
|
||||
- `bootstrap-init`: 应用启动时的集中化初始化机制,包括目录创建、配置加载、验证等
|
||||
- `embedded-config`: 配置嵌入机制,使用 go:embed + 环境变量覆盖,无需外部配置文件
|
||||
|
||||
### Modified Capabilities
|
||||
|
||||
- `dependency-injection`: 调整 bootstrap 流程,在组件初始化前完成目录和配置的准备
|
||||
|
||||
## Impact
|
||||
|
||||
### 代码变更
|
||||
|
||||
| 文件/目录 | 变更类型 | 说明 |
|
||||
|-----------|----------|------|
|
||||
| `pkg/bootstrap/directories.go` | 新增 | 目录初始化逻辑 |
|
||||
| `pkg/config/embedded.go` | 新增 | 配置嵌入和加载逻辑 |
|
||||
| `pkg/config/defaults/config.yaml` | 新增 | 嵌入的默认配置文件 |
|
||||
| `pkg/config/loader.go` | 重写 | 简化为嵌入配置 + 环境变量 |
|
||||
| `pkg/config/watcher.go` | **删除** | 不再需要热重载 |
|
||||
| `pkg/storage/s3.go` | 修改 | 移除目录创建逻辑 |
|
||||
| `cmd/api/main.go` | 修改 | 调用目录初始化 |
|
||||
| `cmd/worker/main.go` | 修改 | 调用目录初始化 |
|
||||
| `internal/bootstrap/` | 修改 | 调整初始化顺序 |
|
||||
| `configs/*.yaml` | **删除** | 配置嵌入后不再需要 |
|
||||
|
||||
### Docker 变更
|
||||
|
||||
| 文件 | 变更类型 | 说明 |
|
||||
|------|----------|------|
|
||||
| `Dockerfile.api` | 修改 | 预创建目录、删除 COPY configs |
|
||||
| `Dockerfile.worker` | 修改 | 预创建目录、删除 COPY configs |
|
||||
| `docker-compose.prod.yml` | 重写 | 纯环境变量配置、删除 configs 挂载 |
|
||||
| `docker/entrypoint-api.sh` | 简化 | 只保留迁移逻辑 |
|
||||
|
||||
### 部署流程变更
|
||||
|
||||
**变更前**(5 步):
|
||||
```bash
|
||||
# 1. SSH 到服务器
|
||||
# 2. 创建目录 mkdir -p /opt/junhong_cmp/{configs,logs}
|
||||
# 3. 复制 docker-compose.prod.yml
|
||||
# 4. 复制配置文件到 configs/
|
||||
# 5. docker-compose up -d
|
||||
```
|
||||
|
||||
**变更后**(1 步):
|
||||
```bash
|
||||
# docker-compose up -d(CI/CD 自动部署,或手动拉取 compose 文件)
|
||||
```
|
||||
|
||||
### 依赖
|
||||
|
||||
- Go 1.16+(`go:embed` 支持)
|
||||
- 无新增外部依赖
|
||||
|
||||
## 预期收益
|
||||
|
||||
| 指标 | 变更前 | 变更后 |
|
||||
|------|--------|--------|
|
||||
| 首次部署步骤 | 5 步 | 1 步 |
|
||||
| 配置文件 | 4 个外部文件 | 0 个(嵌入) |
|
||||
| 权限失败风险 | 高 | 低(降级策略) |
|
||||
| 环境可移植性 | Docker only | Docker/K8s/裸机 |
|
||||
| 配置热重载 | 支持 | 移除(开发阶段不需要) |
|
||||
@@ -0,0 +1,78 @@
|
||||
# bootstrap-init 规范
|
||||
|
||||
应用启动时的集中化初始化机制,确保所有必需目录在组件初始化前创建完成。
|
||||
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: 集中化目录初始化
|
||||
|
||||
系统 SHALL 在应用启动时通过 `bootstrap.EnsureDirectories()` 函数统一创建所有必需的运行时目录。
|
||||
|
||||
目录列表:
|
||||
- 临时文件目录(从 `config.Storage.TempDir` 读取)
|
||||
- 应用日志目录(从 `config.Logging.AppLog.Filename` 提取目录部分)
|
||||
- 访问日志目录(从 `config.Logging.AccessLog.Filename` 提取目录部分)
|
||||
|
||||
#### Scenario: 成功创建所有目录
|
||||
|
||||
- **WHEN** 应用启动且所有目录路径可写
|
||||
- **THEN** 系统创建所有必需目录,权限为 0755
|
||||
- **AND** 函数返回 nil
|
||||
|
||||
#### Scenario: 目录已存在
|
||||
|
||||
- **WHEN** 应用启动且目录已存在
|
||||
- **THEN** 系统跳过创建,不报错
|
||||
- **AND** 函数返回 nil
|
||||
|
||||
#### Scenario: 配置路径为空
|
||||
|
||||
- **WHEN** 某个目录配置为空字符串
|
||||
- **THEN** 系统跳过该目录的创建
|
||||
- **AND** 不影响其他目录的创建
|
||||
|
||||
### Requirement: 权限降级策略
|
||||
|
||||
系统 SHALL 在目录创建权限不足时自动降级到系统临时目录。
|
||||
|
||||
#### Scenario: 权限不足时降级
|
||||
|
||||
- **WHEN** 创建目录因权限不足失败(os.IsPermission 为 true)
|
||||
- **THEN** 系统使用 `os.TempDir()/junhong/<原目录名>` 作为降级路径
|
||||
- **AND** 记录 WARN 级别日志,包含原路径和降级路径
|
||||
- **AND** 函数返回降级后的路径
|
||||
|
||||
#### Scenario: 非权限错误
|
||||
|
||||
- **WHEN** 创建目录失败且不是权限问题
|
||||
- **THEN** 系统返回错误,应用启动失败
|
||||
- **AND** 错误信息包含目录路径和原始错误
|
||||
|
||||
### Requirement: 初始化顺序
|
||||
|
||||
系统 SHALL 确保目录初始化在所有组件初始化之前完成。
|
||||
|
||||
#### Scenario: 正确的初始化顺序
|
||||
|
||||
- **WHEN** 应用启动
|
||||
- **THEN** 执行顺序为:
|
||||
1. config.Load() 加载配置
|
||||
2. bootstrap.EnsureDirectories() 创建目录
|
||||
3. logger.Init() 初始化日志
|
||||
4. 其他组件初始化
|
||||
|
||||
#### Scenario: 目录初始化失败
|
||||
|
||||
- **WHEN** `bootstrap.EnsureDirectories()` 返回错误
|
||||
- **THEN** 应用立即退出,不继续初始化其他组件
|
||||
- **AND** 错误信息输出到 stderr
|
||||
|
||||
### Requirement: 移除分散的目录创建逻辑
|
||||
|
||||
系统 SHALL 移除各组件中分散的目录创建代码。
|
||||
|
||||
#### Scenario: S3Provider 不再创建目录
|
||||
|
||||
- **WHEN** 初始化 S3Provider
|
||||
- **THEN** 不再调用 `os.MkdirAll` 创建临时目录
|
||||
- **AND** 假设目录已由 bootstrap 创建
|
||||
@@ -0,0 +1,133 @@
|
||||
# embedded-config 规范
|
||||
|
||||
配置嵌入机制,使用 go:embed 将默认配置嵌入二进制文件,通过环境变量覆盖。
|
||||
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: 配置嵌入
|
||||
|
||||
系统 SHALL 使用 Go 的 `go:embed` 指令将默认配置文件嵌入二进制文件。
|
||||
|
||||
嵌入文件位置:`pkg/config/defaults/config.yaml`
|
||||
|
||||
#### Scenario: 加载嵌入配置
|
||||
|
||||
- **WHEN** 调用 `config.Load()`
|
||||
- **THEN** 系统从嵌入的 `defaults/config.yaml` 读取默认配置
|
||||
- **AND** 无需外部配置文件即可启动
|
||||
|
||||
#### Scenario: 嵌入配置包含完整结构
|
||||
|
||||
- **WHEN** 读取嵌入配置
|
||||
- **THEN** 配置包含所有配置节:server、database、redis、storage、logging、queue、jwt、middleware
|
||||
|
||||
### Requirement: 环境变量覆盖
|
||||
|
||||
系统 SHALL 支持通过环境变量覆盖嵌入的默认配置值。
|
||||
|
||||
环境变量格式:`JUNHONG_{SECTION}_{KEY}`
|
||||
|
||||
#### Scenario: 环境变量覆盖配置
|
||||
|
||||
- **WHEN** 设置环境变量 `JUNHONG_DATABASE_HOST=myhost`
|
||||
- **THEN** `config.Database.Host` 的值为 "myhost"
|
||||
- **AND** 覆盖嵌入配置中的默认值
|
||||
|
||||
#### Scenario: 嵌套配置覆盖
|
||||
|
||||
- **WHEN** 设置环境变量 `JUNHONG_LOGGING_LEVEL=debug`
|
||||
- **THEN** `config.Logging.Level` 的值为 "debug"
|
||||
|
||||
#### Scenario: 未设置环境变量
|
||||
|
||||
- **WHEN** 未设置某个配置的环境变量
|
||||
- **THEN** 使用嵌入配置中的默认值
|
||||
|
||||
### Requirement: 配置优先级
|
||||
|
||||
系统 SHALL 按以下优先级应用配置(高到低):
|
||||
|
||||
1. 环境变量 (JUNHONG_*)
|
||||
2. 嵌入默认值 (go:embed)
|
||||
|
||||
#### Scenario: 优先级验证
|
||||
|
||||
- **WHEN** 嵌入配置中 `server.address` 为 ":3000"
|
||||
- **AND** 设置环境变量 `JUNHONG_SERVER_ADDRESS=:8080`
|
||||
- **THEN** 最终 `config.Server.Address` 为 ":8080"
|
||||
|
||||
### Requirement: 必填配置验证
|
||||
|
||||
系统 SHALL 在加载配置后验证必填配置项是否已设置。
|
||||
|
||||
必填配置项:
|
||||
- `database.host`
|
||||
- `database.user`
|
||||
- `database.password`
|
||||
- `database.dbname`
|
||||
- `redis.address`
|
||||
- `jwt.secret_key`
|
||||
|
||||
#### Scenario: 必填配置缺失
|
||||
|
||||
- **WHEN** 必填配置项为空且未通过环境变量设置
|
||||
- **THEN** `config.Load()` 返回错误
|
||||
- **AND** 错误信息明确指出缺失的配置项和对应的环境变量名
|
||||
|
||||
#### Scenario: 必填配置通过环境变量提供
|
||||
|
||||
- **WHEN** 所有必填配置通过环境变量设置
|
||||
- **THEN** `config.Load()` 成功返回配置
|
||||
|
||||
### Requirement: 删除外部配置文件支持
|
||||
|
||||
系统 SHALL 移除对外部配置文件的支持。
|
||||
|
||||
#### Scenario: 不读取 configs 目录
|
||||
|
||||
- **WHEN** 应用启动
|
||||
- **THEN** 不读取 `configs/*.yaml` 文件
|
||||
- **AND** 不依赖 `CONFIG_PATH` 或 `CONFIG_ENV` 环境变量
|
||||
|
||||
### Requirement: 删除配置热重载
|
||||
|
||||
系统 SHALL 移除配置热重载功能。
|
||||
|
||||
#### Scenario: 不监听配置文件变化
|
||||
|
||||
- **WHEN** 应用运行中
|
||||
- **THEN** 不使用 fsnotify 监听文件变化
|
||||
- **AND** 删除 `pkg/config/watcher.go`
|
||||
|
||||
#### Scenario: 配置变更需重启
|
||||
|
||||
- **WHEN** 需要更改配置
|
||||
- **THEN** 必须重启应用使新配置生效
|
||||
|
||||
### Requirement: 环境变量前缀
|
||||
|
||||
系统 SHALL 使用 `JUNHONG_` 作为环境变量前缀。
|
||||
|
||||
#### Scenario: 前缀隔离
|
||||
|
||||
- **WHEN** 存在环境变量 `DATABASE_HOST=other`
|
||||
- **AND** 存在环境变量 `JUNHONG_DATABASE_HOST=correct`
|
||||
- **THEN** `config.Database.Host` 为 "correct"
|
||||
- **AND** 忽略无前缀的 `DATABASE_HOST`
|
||||
|
||||
### Requirement: 敏感配置处理
|
||||
|
||||
系统 SHALL 确保敏感配置不嵌入二进制文件。
|
||||
|
||||
敏感配置项(嵌入值为空):
|
||||
- `database.password`
|
||||
- `redis.password`
|
||||
- `jwt.secret_key`
|
||||
- `storage.s3.access_key_id`
|
||||
- `storage.s3.secret_access_key`
|
||||
|
||||
#### Scenario: 敏感配置默认为空
|
||||
|
||||
- **WHEN** 读取嵌入配置
|
||||
- **THEN** 敏感配置项的值为空字符串
|
||||
- **AND** 必须通过环境变量提供实际值
|
||||
@@ -0,0 +1,82 @@
|
||||
# 实施任务清单
|
||||
|
||||
## 1. 配置嵌入模块
|
||||
|
||||
- [x] 1.1 创建 `pkg/config/defaults/config.yaml` 嵌入配置文件
|
||||
- [x] 1.2 创建 `pkg/config/embedded.go`,实现 go:embed 加载逻辑
|
||||
- [x] 1.3 重写 `pkg/config/loader.go`,使用嵌入配置 + 环境变量覆盖
|
||||
- [x] 1.4 更新 `pkg/config/config.go` 中的 Validate() 方法,添加必填配置验证
|
||||
- [x] 1.5 删除 `pkg/config/watcher.go` 配置热重载模块
|
||||
- [x] 1.6 编写配置加载单元测试
|
||||
|
||||
## 2. 目录初始化模块
|
||||
|
||||
- [x] 2.1 创建 `pkg/bootstrap/directories.go`,实现 EnsureDirectories() 函数
|
||||
- [x] 2.2 实现权限降级策略(权限不足时使用临时目录)
|
||||
- [x] 2.3 编写目录初始化单元测试
|
||||
- [x] 2.4 移除 `pkg/storage/s3.go` 中的目录创建逻辑
|
||||
|
||||
## 3. 应用入口改造
|
||||
|
||||
- [x] 3.1 更新 `cmd/api/main.go`,在配置加载后调用 bootstrap.EnsureDirectories()
|
||||
- [x] 3.2 更新 `cmd/worker/main.go`,同样调用目录初始化
|
||||
- [x] 3.3 调整 `internal/bootstrap/` 中的初始化顺序(无需修改,顺序正确)
|
||||
|
||||
## 4. Docker 配置更新
|
||||
|
||||
- [x] 4.1 更新 `Dockerfile.api`:预创建目录、移除 COPY configs
|
||||
- [x] 4.2 更新 `Dockerfile.worker`:预创建目录、移除 COPY configs
|
||||
- [x] 4.3 重写 `docker-compose.prod.yml`:纯环境变量配置
|
||||
- [x] 4.4 简化 `docker/entrypoint-api.sh`:移除配置相关逻辑
|
||||
|
||||
## 5. 清理旧文件
|
||||
|
||||
- [x] 5.1 删除 `configs/config.yaml`
|
||||
- [x] 5.2 删除 `configs/config.dev.yaml`
|
||||
- [x] 5.3 删除 `configs/config.staging.yaml`
|
||||
- [x] 5.4 删除 `configs/config.prod.yaml`
|
||||
- [x] 5.5 删除 `configs/` 目录(如果为空)
|
||||
|
||||
## 6. 文档更新
|
||||
|
||||
- [x] 6.1 更新 README.md 部署说明
|
||||
- [x] 6.2 更新环境变量列表文档(创建 docs/environment-variables.md)
|
||||
- [x] 6.3 更新关键文档(auth-usage-guide, object-storage, add-default-admin-init)
|
||||
|
||||
## 7. 验证
|
||||
|
||||
- [x] 7.1 本地运行测试:`go test ./...`(config 和 bootstrap 测试通过)
|
||||
- [x] 7.2 本地 Docker 构建测试(API 和 Worker 镜像构建成功)
|
||||
- [x] 7.3 本地 docker-compose 启动测试(需要外部 PostgreSQL/Redis,配置验证通过)
|
||||
- [x] 7.4 验证环境变量覆盖功能(TestLoad_EnvOverride 通过)
|
||||
|
||||
---
|
||||
|
||||
## 实施完成总结
|
||||
|
||||
**完成日期**: 2026-01-24
|
||||
|
||||
### 主要变更
|
||||
|
||||
1. **配置嵌入机制**
|
||||
- 默认配置嵌入二进制文件 (`pkg/config/defaults/config.yaml`)
|
||||
- 环境变量覆盖使用 `JUNHONG_` 前缀
|
||||
- 移除配置热重载功能
|
||||
|
||||
2. **目录自动初始化**
|
||||
- `pkg/bootstrap/directories.go` 实现 `EnsureDirectories()`
|
||||
- 支持权限降级(无权限时使用临时目录)
|
||||
|
||||
3. **Docker 部署简化**
|
||||
- 移除 `COPY configs` 指令
|
||||
- 纯环境变量配置
|
||||
- 更新 `docker-compose.prod.yml` 和 `entrypoint-api.sh`
|
||||
|
||||
4. **文档更新**
|
||||
- 创建 `docs/environment-variables.md` 完整环境变量文档
|
||||
- 更新 README.md 部署说明
|
||||
- 更新相关功能文档
|
||||
|
||||
### 后续工作(可选)
|
||||
|
||||
- 更新剩余的旧文档(rate-limiting.md, deployment-guide.md 等)
|
||||
78
openspec/specs/bootstrap-init/spec.md
Normal file
78
openspec/specs/bootstrap-init/spec.md
Normal file
@@ -0,0 +1,78 @@
|
||||
# bootstrap-init Specification
|
||||
|
||||
## Purpose
|
||||
TBD - created by archiving change deployment-self-init. Update Purpose after archive.
|
||||
## Requirements
|
||||
### Requirement: 集中化目录初始化
|
||||
|
||||
系统 SHALL 在应用启动时通过 `bootstrap.EnsureDirectories()` 函数统一创建所有必需的运行时目录。
|
||||
|
||||
目录列表:
|
||||
- 临时文件目录(从 `config.Storage.TempDir` 读取)
|
||||
- 应用日志目录(从 `config.Logging.AppLog.Filename` 提取目录部分)
|
||||
- 访问日志目录(从 `config.Logging.AccessLog.Filename` 提取目录部分)
|
||||
|
||||
#### Scenario: 成功创建所有目录
|
||||
|
||||
- **WHEN** 应用启动且所有目录路径可写
|
||||
- **THEN** 系统创建所有必需目录,权限为 0755
|
||||
- **AND** 函数返回 nil
|
||||
|
||||
#### Scenario: 目录已存在
|
||||
|
||||
- **WHEN** 应用启动且目录已存在
|
||||
- **THEN** 系统跳过创建,不报错
|
||||
- **AND** 函数返回 nil
|
||||
|
||||
#### Scenario: 配置路径为空
|
||||
|
||||
- **WHEN** 某个目录配置为空字符串
|
||||
- **THEN** 系统跳过该目录的创建
|
||||
- **AND** 不影响其他目录的创建
|
||||
|
||||
### Requirement: 权限降级策略
|
||||
|
||||
系统 SHALL 在目录创建权限不足时自动降级到系统临时目录。
|
||||
|
||||
#### Scenario: 权限不足时降级
|
||||
|
||||
- **WHEN** 创建目录因权限不足失败(os.IsPermission 为 true)
|
||||
- **THEN** 系统使用 `os.TempDir()/junhong/<原目录名>` 作为降级路径
|
||||
- **AND** 记录 WARN 级别日志,包含原路径和降级路径
|
||||
- **AND** 函数返回降级后的路径
|
||||
|
||||
#### Scenario: 非权限错误
|
||||
|
||||
- **WHEN** 创建目录失败且不是权限问题
|
||||
- **THEN** 系统返回错误,应用启动失败
|
||||
- **AND** 错误信息包含目录路径和原始错误
|
||||
|
||||
### Requirement: 初始化顺序
|
||||
|
||||
系统 SHALL 确保目录初始化在所有组件初始化之前完成。
|
||||
|
||||
#### Scenario: 正确的初始化顺序
|
||||
|
||||
- **WHEN** 应用启动
|
||||
- **THEN** 执行顺序为:
|
||||
1. config.Load() 加载配置
|
||||
2. bootstrap.EnsureDirectories() 创建目录
|
||||
3. logger.Init() 初始化日志
|
||||
4. 其他组件初始化
|
||||
|
||||
#### Scenario: 目录初始化失败
|
||||
|
||||
- **WHEN** `bootstrap.EnsureDirectories()` 返回错误
|
||||
- **THEN** 应用立即退出,不继续初始化其他组件
|
||||
- **AND** 错误信息输出到 stderr
|
||||
|
||||
### Requirement: 移除分散的目录创建逻辑
|
||||
|
||||
系统 SHALL 移除各组件中分散的目录创建代码。
|
||||
|
||||
#### Scenario: S3Provider 不再创建目录
|
||||
|
||||
- **WHEN** 初始化 S3Provider
|
||||
- **THEN** 不再调用 `os.MkdirAll` 创建临时目录
|
||||
- **AND** 假设目录已由 bootstrap 创建
|
||||
|
||||
133
openspec/specs/embedded-config/spec.md
Normal file
133
openspec/specs/embedded-config/spec.md
Normal file
@@ -0,0 +1,133 @@
|
||||
# embedded-config Specification
|
||||
|
||||
## Purpose
|
||||
TBD - created by archiving change deployment-self-init. Update Purpose after archive.
|
||||
## Requirements
|
||||
### Requirement: 配置嵌入
|
||||
|
||||
系统 SHALL 使用 Go 的 `go:embed` 指令将默认配置文件嵌入二进制文件。
|
||||
|
||||
嵌入文件位置:`pkg/config/defaults/config.yaml`
|
||||
|
||||
#### Scenario: 加载嵌入配置
|
||||
|
||||
- **WHEN** 调用 `config.Load()`
|
||||
- **THEN** 系统从嵌入的 `defaults/config.yaml` 读取默认配置
|
||||
- **AND** 无需外部配置文件即可启动
|
||||
|
||||
#### Scenario: 嵌入配置包含完整结构
|
||||
|
||||
- **WHEN** 读取嵌入配置
|
||||
- **THEN** 配置包含所有配置节:server、database、redis、storage、logging、queue、jwt、middleware
|
||||
|
||||
### Requirement: 环境变量覆盖
|
||||
|
||||
系统 SHALL 支持通过环境变量覆盖嵌入的默认配置值。
|
||||
|
||||
环境变量格式:`JUNHONG_{SECTION}_{KEY}`
|
||||
|
||||
#### Scenario: 环境变量覆盖配置
|
||||
|
||||
- **WHEN** 设置环境变量 `JUNHONG_DATABASE_HOST=myhost`
|
||||
- **THEN** `config.Database.Host` 的值为 "myhost"
|
||||
- **AND** 覆盖嵌入配置中的默认值
|
||||
|
||||
#### Scenario: 嵌套配置覆盖
|
||||
|
||||
- **WHEN** 设置环境变量 `JUNHONG_LOGGING_LEVEL=debug`
|
||||
- **THEN** `config.Logging.Level` 的值为 "debug"
|
||||
|
||||
#### Scenario: 未设置环境变量
|
||||
|
||||
- **WHEN** 未设置某个配置的环境变量
|
||||
- **THEN** 使用嵌入配置中的默认值
|
||||
|
||||
### Requirement: 配置优先级
|
||||
|
||||
系统 SHALL 按以下优先级应用配置(高到低):
|
||||
|
||||
1. 环境变量 (JUNHONG_*)
|
||||
2. 嵌入默认值 (go:embed)
|
||||
|
||||
#### Scenario: 优先级验证
|
||||
|
||||
- **WHEN** 嵌入配置中 `server.address` 为 ":3000"
|
||||
- **AND** 设置环境变量 `JUNHONG_SERVER_ADDRESS=:8080`
|
||||
- **THEN** 最终 `config.Server.Address` 为 ":8080"
|
||||
|
||||
### Requirement: 必填配置验证
|
||||
|
||||
系统 SHALL 在加载配置后验证必填配置项是否已设置。
|
||||
|
||||
必填配置项:
|
||||
- `database.host`
|
||||
- `database.user`
|
||||
- `database.password`
|
||||
- `database.dbname`
|
||||
- `redis.address`
|
||||
- `jwt.secret_key`
|
||||
|
||||
#### Scenario: 必填配置缺失
|
||||
|
||||
- **WHEN** 必填配置项为空且未通过环境变量设置
|
||||
- **THEN** `config.Load()` 返回错误
|
||||
- **AND** 错误信息明确指出缺失的配置项和对应的环境变量名
|
||||
|
||||
#### Scenario: 必填配置通过环境变量提供
|
||||
|
||||
- **WHEN** 所有必填配置通过环境变量设置
|
||||
- **THEN** `config.Load()` 成功返回配置
|
||||
|
||||
### Requirement: 删除外部配置文件支持
|
||||
|
||||
系统 SHALL 移除对外部配置文件的支持。
|
||||
|
||||
#### Scenario: 不读取 configs 目录
|
||||
|
||||
- **WHEN** 应用启动
|
||||
- **THEN** 不读取 `configs/*.yaml` 文件
|
||||
- **AND** 不依赖 `CONFIG_PATH` 或 `CONFIG_ENV` 环境变量
|
||||
|
||||
### Requirement: 删除配置热重载
|
||||
|
||||
系统 SHALL 移除配置热重载功能。
|
||||
|
||||
#### Scenario: 不监听配置文件变化
|
||||
|
||||
- **WHEN** 应用运行中
|
||||
- **THEN** 不使用 fsnotify 监听文件变化
|
||||
- **AND** 删除 `pkg/config/watcher.go`
|
||||
|
||||
#### Scenario: 配置变更需重启
|
||||
|
||||
- **WHEN** 需要更改配置
|
||||
- **THEN** 必须重启应用使新配置生效
|
||||
|
||||
### Requirement: 环境变量前缀
|
||||
|
||||
系统 SHALL 使用 `JUNHONG_` 作为环境变量前缀。
|
||||
|
||||
#### Scenario: 前缀隔离
|
||||
|
||||
- **WHEN** 存在环境变量 `DATABASE_HOST=other`
|
||||
- **AND** 存在环境变量 `JUNHONG_DATABASE_HOST=correct`
|
||||
- **THEN** `config.Database.Host` 为 "correct"
|
||||
- **AND** 忽略无前缀的 `DATABASE_HOST`
|
||||
|
||||
### Requirement: 敏感配置处理
|
||||
|
||||
系统 SHALL 确保敏感配置不嵌入二进制文件。
|
||||
|
||||
敏感配置项(嵌入值为空):
|
||||
- `database.password`
|
||||
- `redis.password`
|
||||
- `jwt.secret_key`
|
||||
- `storage.s3.access_key_id`
|
||||
- `storage.s3.secret_access_key`
|
||||
|
||||
#### Scenario: 敏感配置默认为空
|
||||
|
||||
- **WHEN** 读取嵌入配置
|
||||
- **THEN** 敏感配置项的值为空字符串
|
||||
- **AND** 必须通过环境变量提供实际值
|
||||
|
||||
@@ -16,18 +16,22 @@ TBD - created by archiving change iot-card-standalone-management. Update Purpose
|
||||
|
||||
**导入参数**:
|
||||
- `carrier_id`: 运营商 ID(BIGINT,必填)
|
||||
- `carrier_type`: 运营商类型(VARCHAR(10),CMCC/CUCC/CTCC/CBN)
|
||||
- `batch_no`: 批次号(VARCHAR(100),可选)
|
||||
- `file_name`: 原始文件名(VARCHAR(255),可选)
|
||||
|
||||
**待导入数据**:
|
||||
- `card_list`: 待导入卡列表(JSONB,结构: [{iccid, msisdn}],替代原 iccid_list)
|
||||
|
||||
**进度统计**:
|
||||
- `total_count`: 总数(INT,CSV 文件总行数)
|
||||
- `success_count`: 成功数(INT,成功导入的 ICCID 数量)
|
||||
- `success_count`: 成功数(INT,成功导入的卡数量)
|
||||
- `skip_count`: 跳过数(INT,因重复等原因跳过的数量)
|
||||
- `fail_count`: 失败数(INT,因格式错误等原因失败的数量)
|
||||
|
||||
**结果详情**:
|
||||
- `skipped_items`: 跳过记录详情(JSONB,结构: [{line, iccid, reason}])
|
||||
- `failed_items`: 失败记录详情(JSONB,结构: [{line, iccid, reason}])
|
||||
- `skipped_items`: 跳过记录详情(JSONB,结构: [{line, iccid, msisdn, reason}])
|
||||
- `failed_items`: 失败记录详情(JSONB,结构: [{line, iccid, msisdn, reason}])
|
||||
|
||||
**时间和错误**:
|
||||
- `started_at`: 开始处理时间(TIMESTAMP,可空)
|
||||
@@ -43,23 +47,9 @@ TBD - created by archiving change iot-card-standalone-management. Update Purpose
|
||||
|
||||
#### Scenario: 创建导入任务
|
||||
|
||||
- **WHEN** 管理员上传 CSV 文件发起导入
|
||||
- **THEN** 系统创建导入任务记录,`status` 为 1(待处理),`total_count` 为 CSV 行数,返回任务 ID
|
||||
|
||||
#### Scenario: 导入任务开始处理
|
||||
|
||||
- **WHEN** Worker 开始处理导入任务
|
||||
- **THEN** 系统将任务 `status` 从 1(待处理) 变更为 2(处理中),`started_at` 记录当前时间
|
||||
|
||||
#### Scenario: 导入任务完成
|
||||
|
||||
- **WHEN** Worker 完成导入任务处理
|
||||
- **THEN** 系统将任务 `status` 变更为 3(已完成),`completed_at` 记录当前时间,更新 `success_count`、`skip_count`、`fail_count`
|
||||
|
||||
#### Scenario: 导入任务失败
|
||||
|
||||
- **WHEN** Worker 处理导入任务时发生严重错误(如文件损坏)
|
||||
- **THEN** 系统将任务 `status` 变更为 4(失败),`error_message` 记录错误信息
|
||||
- **GIVEN** 管理员上传包含 ICCID 和 MSISDN 两列的 CSV 文件
|
||||
- **WHEN** 系统解析 CSV 并创建导入任务
|
||||
- **THEN** 系统创建导入任务记录,`card_list` 包含 [{iccid, msisdn}] 结构,`status` 为 1(待处理)
|
||||
|
||||
---
|
||||
|
||||
@@ -174,3 +164,76 @@ TBD - created by archiving change iot-card-standalone-management. Update Purpose
|
||||
- **WHEN** 更新导入任务时 success_count + skip_count + fail_count > total_count
|
||||
- **THEN** 系统拒绝更新,返回错误信息"统计数量不一致"
|
||||
|
||||
### Requirement: CSV 文件格式规范
|
||||
|
||||
系统 SHALL 要求 CSV 文件必须包含 ICCID 和 MSISDN 两列。
|
||||
|
||||
**文件格式要求**:
|
||||
- 第一列: ICCID(必填,不能为空)
|
||||
- 第二列: MSISDN/接入号(必填,不能为空)
|
||||
- 支持表头行(自动识别并跳过)
|
||||
- 表头识别关键字: iccid/卡号 + msisdn/接入号/手机号
|
||||
|
||||
**解析规则**:
|
||||
- 自动去除首尾空格
|
||||
- 跳过空行
|
||||
- 第一行为表头时自动跳过
|
||||
- 列数不足 2 列的文件拒绝导入
|
||||
- ICCID 为空的行记录为失败
|
||||
- MSISDN 为空的行记录为失败
|
||||
|
||||
#### Scenario: 解析标准双列 CSV 文件
|
||||
|
||||
- **GIVEN** CSV 文件内容为:
|
||||
```
|
||||
iccid,msisdn
|
||||
89860012345678901234,13800000001
|
||||
89860012345678901235,13800000002
|
||||
```
|
||||
- **WHEN** 系统解析该 CSV 文件
|
||||
- **THEN** 解析结果包含 2 条有效记录,每条包含 ICCID 和 MSISDN
|
||||
|
||||
#### Scenario: 拒绝单列 CSV 文件
|
||||
|
||||
- **GIVEN** CSV 文件内容仅包含 ICCID 单列
|
||||
- **WHEN** 系统尝试解析该 CSV 文件
|
||||
- **THEN** 系统返回错误 "CSV 文件格式错误:缺少 MSISDN 列"
|
||||
|
||||
#### Scenario: MSISDN 为空的行记录失败
|
||||
|
||||
- **GIVEN** CSV 文件内容为:
|
||||
```
|
||||
iccid,msisdn
|
||||
89860012345678901234,13800000001
|
||||
89860012345678901235,
|
||||
```
|
||||
- **WHEN** 系统解析该 CSV 文件
|
||||
- **THEN** 第一条记录解析成功,第二条记录标记为失败,原因为 "MSISDN 不能为空"
|
||||
|
||||
#### Scenario: ICCID 为空的行记录失败
|
||||
|
||||
- **GIVEN** CSV 文件内容为:
|
||||
```
|
||||
iccid,msisdn
|
||||
89860012345678901234,13800000001
|
||||
,13800000002
|
||||
```
|
||||
- **WHEN** 系统解析该 CSV 文件
|
||||
- **THEN** 第一条记录解析成功,第二条记录标记为失败,原因为 "ICCID 不能为空"
|
||||
|
||||
---
|
||||
|
||||
### Requirement: 导入时填充 MSISDN 字段
|
||||
|
||||
系统 SHALL 在创建 IoT 卡记录时填充 MSISDN 字段。
|
||||
|
||||
**处理规则**:
|
||||
- 从 `card_list` 中获取 ICCID 和 MSISDN
|
||||
- 创建 `IotCard` 记录时同时设置 `iccid` 和 `msisdn` 字段
|
||||
|
||||
#### Scenario: 创建卡记录时填充 MSISDN
|
||||
|
||||
- **GIVEN** 导入任务包含卡数据 [{iccid: "898600...", msisdn: "13800000001"}]
|
||||
- **WHEN** Worker 处理导入任务创建卡记录
|
||||
- **THEN** 创建的 `IotCard` 记录 `iccid` 为 "898600...",`msisdn` 为 "13800000001"
|
||||
|
||||
|
||||
219
openspec/specs/object-storage/spec.md
Normal file
219
openspec/specs/object-storage/spec.md
Normal file
@@ -0,0 +1,219 @@
|
||||
# object-storage Specification
|
||||
|
||||
## Purpose
|
||||
TBD - created by archiving change add-object-storage. Update Purpose after archive.
|
||||
## Requirements
|
||||
### Requirement: Provider 接口
|
||||
|
||||
系统 SHALL 提供统一的对象存储 Provider 接口,支持 S3 兼容的对象存储服务。
|
||||
|
||||
接口定义:
|
||||
```go
|
||||
type Provider interface {
|
||||
Upload(ctx context.Context, key string, reader io.Reader, contentType string) error
|
||||
Download(ctx context.Context, key string, writer io.Writer) error
|
||||
DownloadToTemp(ctx context.Context, key string) (localPath string, cleanup func(), err error)
|
||||
Delete(ctx context.Context, key string) error
|
||||
Exists(ctx context.Context, key string) (bool, error)
|
||||
GetUploadURL(ctx context.Context, key string, contentType string, expires time.Duration) (string, error)
|
||||
GetDownloadURL(ctx context.Context, key string, expires time.Duration) (string, error)
|
||||
}
|
||||
```
|
||||
|
||||
#### Scenario: 创建 S3 Provider
|
||||
- **WHEN** 系统启动时读取 storage 配置
|
||||
- **THEN** 系统 SHALL 创建 S3Provider 实例并验证连接
|
||||
|
||||
#### Scenario: 配置缺失
|
||||
- **WHEN** storage 配置未设置或不完整
|
||||
- **THEN** 系统 SHALL 记录警告日志并跳过初始化(不影响启动)
|
||||
|
||||
---
|
||||
|
||||
### Requirement: 文件上传
|
||||
|
||||
系统 SHALL 支持通过 Provider 接口上传文件到对象存储。
|
||||
|
||||
#### Scenario: 上传成功
|
||||
- **WHEN** 调用 `Upload(ctx, "imports/test.csv", reader, "text/csv")`
|
||||
- **THEN** 文件 SHALL 被上传到配置的 Bucket 中指定路径
|
||||
- **THEN** 方法 SHALL 返回 nil
|
||||
|
||||
#### Scenario: 上传失败
|
||||
- **WHEN** 对象存储服务不可用
|
||||
- **THEN** 方法 SHALL 返回包含错误详情的 error
|
||||
|
||||
---
|
||||
|
||||
### Requirement: 文件下载
|
||||
|
||||
系统 SHALL 支持从对象存储下载文件。
|
||||
|
||||
#### Scenario: 下载到 Writer
|
||||
- **WHEN** 调用 `Download(ctx, "imports/test.csv", writer)`
|
||||
- **THEN** 文件内容 SHALL 被写入到提供的 writer
|
||||
|
||||
#### Scenario: 下载到临时文件
|
||||
- **WHEN** 调用 `DownloadToTemp(ctx, "imports/test.csv")`
|
||||
- **THEN** 系统 SHALL 下载文件到临时目录
|
||||
- **THEN** 方法 SHALL 返回本地文件路径和 cleanup 函数
|
||||
- **THEN** 调用 cleanup() 后临时文件 SHALL 被删除
|
||||
|
||||
#### Scenario: 文件不存在
|
||||
- **WHEN** 下载的文件在对象存储中不存在
|
||||
- **THEN** 方法 SHALL 返回 "文件不存在" 错误
|
||||
|
||||
---
|
||||
|
||||
### Requirement: 文件删除
|
||||
|
||||
系统 SHALL 支持从对象存储删除文件。
|
||||
|
||||
#### Scenario: 删除成功
|
||||
- **WHEN** 调用 `Delete(ctx, "imports/test.csv")`
|
||||
- **THEN** 文件 SHALL 从对象存储中删除
|
||||
- **THEN** 方法 SHALL 返回 nil
|
||||
|
||||
#### Scenario: 删除不存在的文件
|
||||
- **WHEN** 删除的文件不存在
|
||||
- **THEN** 方法 SHALL 返回 nil(幂等操作)
|
||||
|
||||
---
|
||||
|
||||
### Requirement: 文件存在性检查
|
||||
|
||||
系统 SHALL 支持检查文件是否存在于对象存储。
|
||||
|
||||
#### Scenario: 文件存在
|
||||
- **WHEN** 调用 `Exists(ctx, "imports/test.csv")` 且文件存在
|
||||
- **THEN** 方法 SHALL 返回 (true, nil)
|
||||
|
||||
#### Scenario: 文件不存在
|
||||
- **WHEN** 调用 `Exists(ctx, "imports/test.csv")` 且文件不存在
|
||||
- **THEN** 方法 SHALL 返回 (false, nil)
|
||||
|
||||
---
|
||||
|
||||
### Requirement: 预签名上传 URL
|
||||
|
||||
系统 SHALL 支持生成预签名上传 URL,允许前端直接上传文件到对象存储。
|
||||
|
||||
#### Scenario: 生成上传 URL
|
||||
- **WHEN** 调用 `GetUploadURL(ctx, "imports/test.csv", "text/csv", 15*time.Minute)`
|
||||
- **THEN** 方法 SHALL 返回有效的预签名 URL
|
||||
- **THEN** URL SHALL 在指定时间(15分钟)后过期
|
||||
- **THEN** 使用该 URL 的 PUT 请求 SHALL 能成功上传文件
|
||||
|
||||
#### Scenario: URL 过期后
|
||||
- **WHEN** 使用过期的预签名 URL 上传
|
||||
- **THEN** 对象存储 SHALL 返回 403 Forbidden
|
||||
|
||||
---
|
||||
|
||||
### Requirement: 预签名下载 URL
|
||||
|
||||
系统 SHALL 支持生成预签名下载 URL,允许用户直接从对象存储下载文件。
|
||||
|
||||
#### Scenario: 生成下载 URL
|
||||
- **WHEN** 调用 `GetDownloadURL(ctx, "exports/report.xlsx", 24*time.Hour)`
|
||||
- **THEN** 方法 SHALL 返回有效的预签名 URL
|
||||
- **THEN** URL SHALL 在指定时间(24小时)后过期
|
||||
- **THEN** 使用该 URL 的 GET 请求 SHALL 能下载文件
|
||||
|
||||
---
|
||||
|
||||
### Requirement: 获取上传 URL API
|
||||
|
||||
系统 SHALL 提供 API 接口供前端获取预签名上传 URL。
|
||||
|
||||
接口定义:
|
||||
```
|
||||
POST /api/admin/storage/upload-url
|
||||
Authorization: Bearer <token>
|
||||
Content-Type: application/json
|
||||
|
||||
Request:
|
||||
{
|
||||
"file_name": "cards.csv",
|
||||
"content_type": "text/csv",
|
||||
"purpose": "iot_import"
|
||||
}
|
||||
|
||||
Response:
|
||||
{
|
||||
"code": 0,
|
||||
"message": "success",
|
||||
"data": {
|
||||
"upload_url": "http://obs-helf.cucloud.cn/cmp/imports/2025/01/24/abc123.csv?X-Amz-...",
|
||||
"file_key": "imports/2025/01/24/abc123.csv",
|
||||
"expires_in": 900
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
#### Scenario: 获取上传 URL 成功
|
||||
- **WHEN** 已认证用户调用 POST /api/admin/storage/upload-url
|
||||
- **AND** 请求包含有效的 file_name、content_type、purpose
|
||||
- **THEN** 系统 SHALL 返回预签名上传 URL 和 file_key
|
||||
- **THEN** file_key 格式 SHALL 为 `{purpose}/{year}/{month}/{day}/{uuid}.{ext}`
|
||||
|
||||
#### Scenario: 参数缺失
|
||||
- **WHEN** 请求缺少必填参数
|
||||
- **THEN** 系统 SHALL 返回 400 错误
|
||||
|
||||
#### Scenario: 未认证
|
||||
- **WHEN** 请求未携带有效 Token
|
||||
- **THEN** 系统 SHALL 返回 401 错误
|
||||
|
||||
---
|
||||
|
||||
### Requirement: 文件路径规范
|
||||
|
||||
系统 SHALL 按照规范生成文件路径。
|
||||
|
||||
路径格式:`{purpose}/{year}/{month}/{day}/{uuid}.{ext}`
|
||||
|
||||
支持的 purpose 值:
|
||||
- `iot_import` → `imports/`
|
||||
- `export` → `exports/`
|
||||
- `attachment` → `attachments/`
|
||||
|
||||
#### Scenario: 生成导入文件路径
|
||||
- **WHEN** purpose 为 "iot_import",file_name 为 "cards.csv"
|
||||
- **THEN** 生成的 file_key SHALL 匹配 `imports/\d{4}/\d{2}/\d{2}/[a-f0-9-]+\.csv`
|
||||
|
||||
#### Scenario: 未知 purpose
|
||||
- **WHEN** purpose 值不在支持列表中
|
||||
- **THEN** 系统 SHALL 返回错误 "不支持的文件用途"
|
||||
|
||||
---
|
||||
|
||||
### Requirement: 配置结构
|
||||
|
||||
系统 SHALL 支持通过配置文件配置对象存储参数。
|
||||
|
||||
```yaml
|
||||
storage:
|
||||
provider: "s3"
|
||||
s3:
|
||||
endpoint: "http://obs-helf.cucloud.cn"
|
||||
region: "cn-langfang-2"
|
||||
bucket: "cmp"
|
||||
access_key_id: "${OSS_ACCESS_KEY_ID}"
|
||||
secret_access_key: "${OSS_SECRET_ACCESS_KEY}"
|
||||
use_ssl: false
|
||||
path_style: true
|
||||
presign:
|
||||
upload_expires: "15m"
|
||||
download_expires: "24h"
|
||||
temp_dir: "/tmp/junhong-storage"
|
||||
```
|
||||
|
||||
#### Scenario: 环境变量替换
|
||||
- **WHEN** 配置值为 `${ENV_VAR}` 格式
|
||||
- **THEN** 系统 SHALL 从环境变量读取实际值
|
||||
|
||||
#### Scenario: 临时目录不存在
|
||||
- **WHEN** temp_dir 目录不存在
|
||||
- **THEN** 系统 SHALL 自动创建该目录
|
||||
|
||||
60
openspec/specs/openapi-markdown-description/spec.md
Normal file
60
openspec/specs/openapi-markdown-description/spec.md
Normal file
@@ -0,0 +1,60 @@
|
||||
# openapi-markdown-description Specification
|
||||
|
||||
## Purpose
|
||||
TBD - created by archiving change add-openapi-markdown-description. Update Purpose after archive.
|
||||
## Requirements
|
||||
### Requirement: RouteSpec 支持 Description 字段
|
||||
|
||||
RouteSpec 结构体 SHALL 包含 `Description` 字段,类型为 `string`,用于设置接口的详细 Markdown 说明。
|
||||
|
||||
#### Scenario: Description 字段为空时不影响生成
|
||||
|
||||
- **WHEN** RouteSpec.Description 为空字符串
|
||||
- **THEN** 生成的 OpenAPI 规范中该接口不包含 description 字段
|
||||
|
||||
#### Scenario: Description 字段有内容时写入 OpenAPI
|
||||
|
||||
- **WHEN** RouteSpec.Description 包含非空内容
|
||||
- **THEN** 生成的 OpenAPI 规范中该接口的 description 字段包含该内容
|
||||
|
||||
### Requirement: Description 支持 Markdown 语法
|
||||
|
||||
生成器 SHALL 原样保留 Description 字段的 Markdown 内容,不进行转义或处理,以便 OpenAPI 工具(如 Apifox)正确渲染。
|
||||
|
||||
#### Scenario: 支持基础 Markdown 格式
|
||||
|
||||
- **WHEN** Description 包含 Markdown 标题、列表、表格、代码块
|
||||
- **THEN** 生成的 OpenAPI YAML 文件中保留完整的 Markdown 格式
|
||||
|
||||
#### Scenario: 支持多行内容
|
||||
|
||||
- **WHEN** Description 包含多行文本
|
||||
- **THEN** 生成的 OpenAPI YAML 文件使用 YAML 多行字符串格式正确表示
|
||||
|
||||
### Requirement: AddOperation 方法处理 Description
|
||||
|
||||
AddOperation 方法 SHALL 接受 description 参数并设置到 openapi3.Operation.Description 字段。
|
||||
|
||||
#### Scenario: 普通接口设置 Description
|
||||
|
||||
- **WHEN** 调用 AddOperation 且 description 参数非空
|
||||
- **THEN** 生成的 Operation 对象包含 Description 字段
|
||||
|
||||
### Requirement: AddMultipartOperation 方法处理 Description
|
||||
|
||||
AddMultipartOperation 方法 SHALL 与 AddOperation 一致,支持 description 参数。
|
||||
|
||||
#### Scenario: 文件上传接口设置 Description
|
||||
|
||||
- **WHEN** 调用 AddMultipartOperation 且 description 参数非空
|
||||
- **THEN** 生成的 multipart/form-data 接口包含 Description 字段
|
||||
|
||||
### Requirement: Register 函数传递 Description
|
||||
|
||||
Register 函数 SHALL 从 RouteSpec 中提取 Description 字段并传递给文档生成器。
|
||||
|
||||
#### Scenario: Register 调用时传递 Description
|
||||
|
||||
- **WHEN** 调用 Register 函数注册路由
|
||||
- **THEN** RouteSpec.Description 被传递到对应的 AddOperation 或 AddMultipartOperation 调用
|
||||
|
||||
69
pkg/bootstrap/directories.go
Normal file
69
pkg/bootstrap/directories.go
Normal file
@@ -0,0 +1,69 @@
|
||||
package bootstrap
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
|
||||
"github.com/break/junhong_cmp_fiber/pkg/config"
|
||||
"go.uber.org/zap"
|
||||
)
|
||||
|
||||
type DirectoryResult struct {
|
||||
TempDir string
|
||||
AppLogDir string
|
||||
AccessLogDir string
|
||||
Fallbacks []string
|
||||
}
|
||||
|
||||
func EnsureDirectories(cfg *config.Config, logger *zap.Logger) (*DirectoryResult, error) {
|
||||
result := &DirectoryResult{}
|
||||
|
||||
directories := []struct {
|
||||
path string
|
||||
configKey string
|
||||
resultPtr *string
|
||||
}{
|
||||
{cfg.Storage.TempDir, "storage.temp_dir", &result.TempDir},
|
||||
{filepath.Dir(cfg.Logging.AppLog.Filename), "logging.app_log.filename", &result.AppLogDir},
|
||||
{filepath.Dir(cfg.Logging.AccessLog.Filename), "logging.access_log.filename", &result.AccessLogDir},
|
||||
}
|
||||
|
||||
for _, dir := range directories {
|
||||
if dir.path == "" || dir.path == "." {
|
||||
continue
|
||||
}
|
||||
|
||||
actualPath, fallback, err := ensureDirectory(dir.path, logger)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("创建目录 %s (%s) 失败: %w", dir.path, dir.configKey, err)
|
||||
}
|
||||
|
||||
*dir.resultPtr = actualPath
|
||||
if fallback {
|
||||
result.Fallbacks = append(result.Fallbacks, actualPath)
|
||||
}
|
||||
}
|
||||
|
||||
return result, nil
|
||||
}
|
||||
|
||||
func ensureDirectory(path string, logger *zap.Logger) (actualPath string, fallback bool, err error) {
|
||||
if err := os.MkdirAll(path, 0755); err != nil {
|
||||
if os.IsPermission(err) {
|
||||
fallbackPath := filepath.Join(os.TempDir(), "junhong", filepath.Base(path))
|
||||
if mkErr := os.MkdirAll(fallbackPath, 0755); mkErr != nil {
|
||||
return "", false, fmt.Errorf("原路径 %s 权限不足,降级路径 %s 也创建失败: %w", path, fallbackPath, mkErr)
|
||||
}
|
||||
if logger != nil {
|
||||
logger.Warn("目录权限不足,使用降级路径",
|
||||
zap.String("original", path),
|
||||
zap.String("fallback", fallbackPath),
|
||||
)
|
||||
}
|
||||
return fallbackPath, true, nil
|
||||
}
|
||||
return "", false, err
|
||||
}
|
||||
return path, false, nil
|
||||
}
|
||||
100
pkg/bootstrap/directories_test.go
Normal file
100
pkg/bootstrap/directories_test.go
Normal file
@@ -0,0 +1,100 @@
|
||||
package bootstrap
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
|
||||
"github.com/break/junhong_cmp_fiber/pkg/config"
|
||||
)
|
||||
|
||||
func TestEnsureDirectories_Success(t *testing.T) {
|
||||
tmpDir := t.TempDir()
|
||||
cfg := &config.Config{
|
||||
Storage: config.StorageConfig{
|
||||
TempDir: filepath.Join(tmpDir, "storage"),
|
||||
},
|
||||
Logging: config.LoggingConfig{
|
||||
AppLog: config.LogRotationConfig{Filename: filepath.Join(tmpDir, "logs", "app.log")},
|
||||
AccessLog: config.LogRotationConfig{Filename: filepath.Join(tmpDir, "logs", "access.log")},
|
||||
},
|
||||
}
|
||||
|
||||
result, err := EnsureDirectories(cfg, nil)
|
||||
if err != nil {
|
||||
t.Fatalf("EnsureDirectories() 失败: %v", err)
|
||||
}
|
||||
|
||||
if result.TempDir != cfg.Storage.TempDir {
|
||||
t.Errorf("TempDir 期望 %s, 实际 %s", cfg.Storage.TempDir, result.TempDir)
|
||||
}
|
||||
if result.AppLogDir != filepath.Join(tmpDir, "logs") {
|
||||
t.Errorf("AppLogDir 期望 %s, 实际 %s", filepath.Join(tmpDir, "logs"), result.AppLogDir)
|
||||
}
|
||||
|
||||
if _, err := os.Stat(result.TempDir); os.IsNotExist(err) {
|
||||
t.Error("TempDir 目录未创建")
|
||||
}
|
||||
if _, err := os.Stat(result.AppLogDir); os.IsNotExist(err) {
|
||||
t.Error("AppLogDir 目录未创建")
|
||||
}
|
||||
}
|
||||
|
||||
func TestEnsureDirectories_ExistingDirs(t *testing.T) {
|
||||
tmpDir := t.TempDir()
|
||||
storageDir := filepath.Join(tmpDir, "storage")
|
||||
os.MkdirAll(storageDir, 0755)
|
||||
|
||||
cfg := &config.Config{
|
||||
Storage: config.StorageConfig{TempDir: storageDir},
|
||||
Logging: config.LoggingConfig{
|
||||
AppLog: config.LogRotationConfig{Filename: filepath.Join(tmpDir, "logs", "app.log")},
|
||||
AccessLog: config.LogRotationConfig{Filename: filepath.Join(tmpDir, "logs", "access.log")},
|
||||
},
|
||||
}
|
||||
|
||||
result, err := EnsureDirectories(cfg, nil)
|
||||
if err != nil {
|
||||
t.Fatalf("EnsureDirectories() 失败: %v", err)
|
||||
}
|
||||
|
||||
if result.TempDir != storageDir {
|
||||
t.Errorf("已存在目录应返回原路径")
|
||||
}
|
||||
}
|
||||
|
||||
func TestEnsureDirectories_EmptyPaths(t *testing.T) {
|
||||
cfg := &config.Config{
|
||||
Storage: config.StorageConfig{TempDir: ""},
|
||||
Logging: config.LoggingConfig{
|
||||
AppLog: config.LogRotationConfig{Filename: ""},
|
||||
AccessLog: config.LogRotationConfig{Filename: ""},
|
||||
},
|
||||
}
|
||||
|
||||
result, err := EnsureDirectories(cfg, nil)
|
||||
if err != nil {
|
||||
t.Fatalf("EnsureDirectories() 空路径时不应失败: %v", err)
|
||||
}
|
||||
|
||||
if len(result.Fallbacks) != 0 {
|
||||
t.Error("空路径不应产生降级")
|
||||
}
|
||||
}
|
||||
|
||||
func TestEnsureDirectory_Fallback(t *testing.T) {
|
||||
path, fallback, err := ensureDirectory("/root/no_permission_dir_test_"+t.Name(), nil)
|
||||
if err != nil {
|
||||
if os.Getuid() == 0 {
|
||||
t.Skip("以 root 身份运行,跳过权限测试")
|
||||
}
|
||||
t.Skip("无法测试权限降级场景")
|
||||
}
|
||||
|
||||
if fallback {
|
||||
if !filepath.HasPrefix(path, os.TempDir()) {
|
||||
t.Errorf("降级路径应在临时目录下,实际: %s", path)
|
||||
}
|
||||
os.RemoveAll(path)
|
||||
}
|
||||
}
|
||||
@@ -3,6 +3,7 @@ package config
|
||||
import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"strings"
|
||||
"sync/atomic"
|
||||
"time"
|
||||
)
|
||||
@@ -21,6 +22,7 @@ type Config struct {
|
||||
SMS SMSConfig `mapstructure:"sms"`
|
||||
JWT JWTConfig `mapstructure:"jwt"`
|
||||
DefaultAdmin DefaultAdminConfig `mapstructure:"default_admin"`
|
||||
Storage StorageConfig `mapstructure:"storage"`
|
||||
}
|
||||
|
||||
// ServerConfig HTTP 服务器配置
|
||||
@@ -120,7 +122,61 @@ type DefaultAdminConfig struct {
|
||||
Phone string `mapstructure:"phone"`
|
||||
}
|
||||
|
||||
// Validate 验证配置值
|
||||
// StorageConfig 对象存储配置
|
||||
type StorageConfig struct {
|
||||
Provider string `mapstructure:"provider"` // 存储提供商:s3
|
||||
S3 S3Config `mapstructure:"s3"` // S3 兼容存储配置
|
||||
Presign PresignConfig `mapstructure:"presign"` // 预签名 URL 配置
|
||||
TempDir string `mapstructure:"temp_dir"` // 临时文件目录
|
||||
}
|
||||
|
||||
// S3Config S3 兼容存储配置
|
||||
type S3Config struct {
|
||||
Endpoint string `mapstructure:"endpoint"` // 服务端点(如:http://obs-helf.cucloud.cn)
|
||||
Region string `mapstructure:"region"` // 区域(如:cn-langfang-2)
|
||||
Bucket string `mapstructure:"bucket"` // 存储桶名称
|
||||
AccessKeyID string `mapstructure:"access_key_id"` // 访问密钥 ID
|
||||
SecretAccessKey string `mapstructure:"secret_access_key"` // 访问密钥
|
||||
UseSSL bool `mapstructure:"use_ssl"` // 是否使用 SSL
|
||||
PathStyle bool `mapstructure:"path_style"` // 是否使用路径风格(兼容性)
|
||||
}
|
||||
|
||||
// PresignConfig 预签名 URL 配置
|
||||
type PresignConfig struct {
|
||||
UploadExpires time.Duration `mapstructure:"upload_expires"` // 上传 URL 有效期(默认:15m)
|
||||
DownloadExpires time.Duration `mapstructure:"download_expires"` // 下载 URL 有效期(默认:24h)
|
||||
}
|
||||
|
||||
type requiredField struct {
|
||||
value string
|
||||
name string
|
||||
envName string
|
||||
}
|
||||
|
||||
func (c *Config) ValidateRequired() error {
|
||||
fields := []requiredField{
|
||||
{c.Database.Host, "database.host", "JUNHONG_DATABASE_HOST"},
|
||||
{c.Database.User, "database.user", "JUNHONG_DATABASE_USER"},
|
||||
{c.Database.Password, "database.password", "JUNHONG_DATABASE_PASSWORD"},
|
||||
{c.Database.DBName, "database.dbname", "JUNHONG_DATABASE_DBNAME"},
|
||||
{c.Redis.Address, "redis.address", "JUNHONG_REDIS_ADDRESS"},
|
||||
{c.JWT.SecretKey, "jwt.secret_key", "JUNHONG_JWT_SECRET_KEY"},
|
||||
}
|
||||
|
||||
var missing []string
|
||||
for _, f := range fields {
|
||||
if f.value == "" {
|
||||
missing = append(missing, fmt.Sprintf(" - %s (环境变量: %s)", f.name, f.envName))
|
||||
}
|
||||
}
|
||||
|
||||
if len(missing) > 0 {
|
||||
return fmt.Errorf("缺少必填配置项:\n%s", strings.Join(missing, "\n"))
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
func (c *Config) Validate() error {
|
||||
// 服务器验证
|
||||
if c.Server.Address == "" {
|
||||
@@ -184,28 +240,24 @@ func (c *Config) Validate() error {
|
||||
return fmt.Errorf("invalid configuration: middleware.rate_limiter.storage: invalid storage type (current value: %s, expected: memory or redis)", c.Middleware.RateLimiter.Storage)
|
||||
}
|
||||
|
||||
// 短信服务验证
|
||||
if c.SMS.GatewayURL == "" {
|
||||
return fmt.Errorf("invalid configuration: sms.gateway_url: must be non-empty (current value: empty)")
|
||||
}
|
||||
if c.SMS.Username == "" {
|
||||
return fmt.Errorf("invalid configuration: sms.username: must be non-empty (current value: empty)")
|
||||
}
|
||||
if c.SMS.Password == "" {
|
||||
return fmt.Errorf("invalid configuration: sms.password: must be non-empty (current value: empty)")
|
||||
}
|
||||
if c.SMS.Signature == "" {
|
||||
return fmt.Errorf("invalid configuration: sms.signature: must be non-empty (current value: empty)")
|
||||
}
|
||||
if c.SMS.Timeout < 5*time.Second || c.SMS.Timeout > 60*time.Second {
|
||||
return fmt.Errorf("invalid configuration: sms.timeout: duration out of range (current value: %s, expected: 5s-60s)", c.SMS.Timeout)
|
||||
// 短信服务验证(可选,配置 GatewayURL 时才验证其他字段)
|
||||
if c.SMS.GatewayURL != "" {
|
||||
if c.SMS.Username == "" {
|
||||
return fmt.Errorf("invalid configuration: sms.username: must be non-empty when gateway_url is configured")
|
||||
}
|
||||
if c.SMS.Password == "" {
|
||||
return fmt.Errorf("invalid configuration: sms.password: must be non-empty when gateway_url is configured")
|
||||
}
|
||||
if c.SMS.Signature == "" {
|
||||
return fmt.Errorf("invalid configuration: sms.signature: must be non-empty when gateway_url is configured")
|
||||
}
|
||||
if c.SMS.Timeout > 0 && (c.SMS.Timeout < 5*time.Second || c.SMS.Timeout > 60*time.Second) {
|
||||
return fmt.Errorf("invalid configuration: sms.timeout: duration out of range (current value: %s, expected: 5s-60s)", c.SMS.Timeout)
|
||||
}
|
||||
}
|
||||
|
||||
// JWT 验证
|
||||
if c.JWT.SecretKey == "" {
|
||||
return fmt.Errorf("invalid configuration: jwt.secret_key: must be non-empty (current value: empty)")
|
||||
}
|
||||
if len(c.JWT.SecretKey) < 32 {
|
||||
// JWT 验证(SecretKey 必填验证在 ValidateRequired 中处理)
|
||||
if len(c.JWT.SecretKey) > 0 && len(c.JWT.SecretKey) < 32 {
|
||||
return fmt.Errorf("invalid configuration: jwt.secret_key: secret key too short (current length: %d, expected: >= 32)", len(c.JWT.SecretKey))
|
||||
}
|
||||
if c.JWT.TokenDuration < 1*time.Hour || c.JWT.TokenDuration > 720*time.Hour {
|
||||
|
||||
@@ -51,6 +51,11 @@ func TestConfig_Validate(t *testing.T) {
|
||||
Storage: "memory",
|
||||
},
|
||||
},
|
||||
JWT: JWTConfig{
|
||||
TokenDuration: 24 * time.Hour,
|
||||
AccessTokenTTL: 24 * time.Hour,
|
||||
RefreshTokenTTL: 168 * time.Hour,
|
||||
},
|
||||
},
|
||||
wantErr: false,
|
||||
},
|
||||
@@ -582,6 +587,11 @@ func TestSet(t *testing.T) {
|
||||
MaxSize: 500,
|
||||
},
|
||||
},
|
||||
JWT: JWTConfig{
|
||||
TokenDuration: 24 * time.Hour,
|
||||
AccessTokenTTL: 24 * time.Hour,
|
||||
RefreshTokenTTL: 168 * time.Hour,
|
||||
},
|
||||
}
|
||||
|
||||
err := Set(validCfg)
|
||||
|
||||
106
pkg/config/defaults/config.yaml
Normal file
106
pkg/config/defaults/config.yaml
Normal file
@@ -0,0 +1,106 @@
|
||||
# 默认配置文件(嵌入二进制)
|
||||
# 敏感配置和必填配置为空,必须通过环境变量设置
|
||||
# 环境变量格式:JUNHONG_{SECTION}_{KEY}
|
||||
|
||||
server:
|
||||
address: ":3000"
|
||||
read_timeout: "30s"
|
||||
write_timeout: "30s"
|
||||
shutdown_timeout: "30s"
|
||||
prefork: false
|
||||
|
||||
# 数据库配置(必填项需通过环境变量设置)
|
||||
database:
|
||||
host: "" # 必填:JUNHONG_DATABASE_HOST
|
||||
port: 5432
|
||||
user: "" # 必填:JUNHONG_DATABASE_USER
|
||||
password: "" # 必填:JUNHONG_DATABASE_PASSWORD(敏感)
|
||||
dbname: "" # 必填:JUNHONG_DATABASE_DBNAME
|
||||
sslmode: "disable"
|
||||
max_open_conns: 25
|
||||
max_idle_conns: 10
|
||||
conn_max_lifetime: "5m"
|
||||
|
||||
# Redis 配置(必填项需通过环境变量设置)
|
||||
redis:
|
||||
address: "" # 必填:JUNHONG_REDIS_ADDRESS
|
||||
port: 6379
|
||||
password: "" # 可选:JUNHONG_REDIS_PASSWORD(敏感)
|
||||
db: 0
|
||||
pool_size: 10
|
||||
min_idle_conns: 5
|
||||
dial_timeout: "5s"
|
||||
read_timeout: "3s"
|
||||
write_timeout: "3s"
|
||||
|
||||
# 对象存储配置
|
||||
storage:
|
||||
provider: "s3"
|
||||
temp_dir: "/tmp/junhong-storage"
|
||||
s3:
|
||||
endpoint: "" # 可选:JUNHONG_STORAGE_S3_ENDPOINT
|
||||
region: "" # 可选:JUNHONG_STORAGE_S3_REGION
|
||||
bucket: "" # 可选:JUNHONG_STORAGE_S3_BUCKET
|
||||
access_key_id: "" # 可选:JUNHONG_STORAGE_S3_ACCESS_KEY_ID(敏感)
|
||||
secret_access_key: "" # 可选:JUNHONG_STORAGE_S3_SECRET_ACCESS_KEY(敏感)
|
||||
use_ssl: false
|
||||
path_style: true
|
||||
presign:
|
||||
upload_expires: "15m"
|
||||
download_expires: "24h"
|
||||
|
||||
# 日志配置
|
||||
logging:
|
||||
level: "info"
|
||||
development: false
|
||||
app_log:
|
||||
filename: "/app/logs/app.log"
|
||||
max_size: 100
|
||||
max_backups: 3
|
||||
max_age: 7
|
||||
compress: true
|
||||
access_log:
|
||||
filename: "/app/logs/access.log"
|
||||
max_size: 100
|
||||
max_backups: 3
|
||||
max_age: 7
|
||||
compress: true
|
||||
|
||||
# 任务队列配置
|
||||
queue:
|
||||
concurrency: 10
|
||||
queues:
|
||||
critical: 6
|
||||
default: 3
|
||||
low: 1
|
||||
retry_max: 5
|
||||
timeout: "10m"
|
||||
|
||||
# JWT 配置(必填项需通过环境变量设置)
|
||||
jwt:
|
||||
secret_key: "" # 必填:JUNHONG_JWT_SECRET_KEY(敏感)
|
||||
token_duration: "24h"
|
||||
access_token_ttl: "24h"
|
||||
refresh_token_ttl: "168h"
|
||||
|
||||
# 中间件配置
|
||||
middleware:
|
||||
enable_rate_limiter: false
|
||||
rate_limiter:
|
||||
max: 100
|
||||
expiration: "1m"
|
||||
storage: "memory"
|
||||
|
||||
# 短信服务配置
|
||||
sms:
|
||||
gateway_url: "" # 可选:JUNHONG_SMS_GATEWAY_URL
|
||||
username: "" # 可选:JUNHONG_SMS_USERNAME
|
||||
password: "" # 可选:JUNHONG_SMS_PASSWORD(敏感)
|
||||
signature: "" # 可选:JUNHONG_SMS_SIGNATURE
|
||||
timeout: "10s"
|
||||
|
||||
# 默认超级管理员配置(可选)
|
||||
default_admin:
|
||||
username: ""
|
||||
password: ""
|
||||
phone: ""
|
||||
17
pkg/config/embedded.go
Normal file
17
pkg/config/embedded.go
Normal file
@@ -0,0 +1,17 @@
|
||||
package config
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"embed"
|
||||
)
|
||||
|
||||
//go:embed defaults/config.yaml
|
||||
var defaultConfigFS embed.FS
|
||||
|
||||
func getEmbeddedConfig() (*bytes.Reader, error) {
|
||||
data, err := defaultConfigFS.ReadFile("defaults/config.yaml")
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return bytes.NewReader(data), nil
|
||||
}
|
||||
@@ -2,92 +2,120 @@ package config
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
|
||||
"github.com/break/junhong_cmp_fiber/pkg/constants"
|
||||
"github.com/spf13/viper"
|
||||
)
|
||||
|
||||
// Load 从文件和环境变量加载配置
|
||||
const envPrefix = "JUNHONG"
|
||||
|
||||
func Load() (*Config, error) {
|
||||
// 确定配置路径
|
||||
configPath := os.Getenv(constants.EnvConfigPath)
|
||||
if configPath == "" {
|
||||
configPath = constants.DefaultConfigPath
|
||||
}
|
||||
v := viper.New()
|
||||
|
||||
// 检查环境特定配置(dev, staging, prod)
|
||||
configEnv := os.Getenv(constants.EnvConfigEnv)
|
||||
if configEnv != "" {
|
||||
// 优先尝试环境特定配置
|
||||
envConfigPath := fmt.Sprintf("configs/config.%s.yaml", configEnv)
|
||||
if _, err := os.Stat(envConfigPath); err == nil {
|
||||
configPath = envConfigPath
|
||||
}
|
||||
}
|
||||
|
||||
// 设置 Viper
|
||||
viper.SetConfigFile(configPath)
|
||||
viper.SetConfigType("yaml")
|
||||
|
||||
// 启用环境变量覆盖
|
||||
viper.AutomaticEnv()
|
||||
viper.SetEnvPrefix("APP")
|
||||
|
||||
// 读取配置文件
|
||||
if err := viper.ReadInConfig(); err != nil {
|
||||
return nil, fmt.Errorf("failed to read config file: %w", err)
|
||||
}
|
||||
|
||||
// 反序列化到 Config 结构体
|
||||
cfg := &Config{}
|
||||
if err := viper.Unmarshal(cfg); err != nil {
|
||||
return nil, fmt.Errorf("failed to unmarshal config: %w", err)
|
||||
}
|
||||
|
||||
// 验证配置
|
||||
if err := cfg.Validate(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// 设为全局配置
|
||||
globalConfig.Store(cfg)
|
||||
|
||||
return cfg, nil
|
||||
}
|
||||
|
||||
// Reload 重新加载当前配置文件
|
||||
func Reload() (*Config, error) {
|
||||
if err := viper.ReadInConfig(); err != nil {
|
||||
return nil, fmt.Errorf("failed to reload config: %w", err)
|
||||
}
|
||||
|
||||
cfg := &Config{}
|
||||
if err := viper.Unmarshal(cfg); err != nil {
|
||||
return nil, fmt.Errorf("failed to unmarshal config: %w", err)
|
||||
}
|
||||
|
||||
// 设置前验证
|
||||
if err := cfg.Validate(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// 原子交换
|
||||
globalConfig.Store(cfg)
|
||||
|
||||
return cfg, nil
|
||||
}
|
||||
|
||||
// GetConfigPath 返回当前已加载配置文件的绝对路径
|
||||
func GetConfigPath() string {
|
||||
configFile := viper.ConfigFileUsed()
|
||||
if configFile == "" {
|
||||
return ""
|
||||
}
|
||||
absPath, err := filepath.Abs(configFile)
|
||||
embeddedReader, err := getEmbeddedConfig()
|
||||
if err != nil {
|
||||
return configFile
|
||||
return nil, fmt.Errorf("读取嵌入配置失败: %w", err)
|
||||
}
|
||||
|
||||
v.SetConfigType("yaml")
|
||||
if err := v.ReadConfig(embeddedReader); err != nil {
|
||||
return nil, fmt.Errorf("解析嵌入配置失败: %w", err)
|
||||
}
|
||||
|
||||
v.SetEnvPrefix(envPrefix)
|
||||
v.SetEnvKeyReplacer(strings.NewReplacer(".", "_"))
|
||||
v.AutomaticEnv()
|
||||
|
||||
bindEnvVariables(v)
|
||||
|
||||
cfg := &Config{}
|
||||
if err := v.Unmarshal(cfg); err != nil {
|
||||
return nil, fmt.Errorf("反序列化配置失败: %w", err)
|
||||
}
|
||||
|
||||
if err := cfg.ValidateRequired(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if err := cfg.Validate(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
globalConfig.Store(cfg)
|
||||
|
||||
return cfg, nil
|
||||
}
|
||||
|
||||
func bindEnvVariables(v *viper.Viper) {
|
||||
bindings := []string{
|
||||
"server.address",
|
||||
"server.read_timeout",
|
||||
"server.write_timeout",
|
||||
"server.shutdown_timeout",
|
||||
"server.prefork",
|
||||
"database.host",
|
||||
"database.port",
|
||||
"database.user",
|
||||
"database.password",
|
||||
"database.dbname",
|
||||
"database.sslmode",
|
||||
"database.max_open_conns",
|
||||
"database.max_idle_conns",
|
||||
"database.conn_max_lifetime",
|
||||
"redis.address",
|
||||
"redis.port",
|
||||
"redis.password",
|
||||
"redis.db",
|
||||
"redis.pool_size",
|
||||
"redis.min_idle_conns",
|
||||
"redis.dial_timeout",
|
||||
"redis.read_timeout",
|
||||
"redis.write_timeout",
|
||||
"storage.provider",
|
||||
"storage.temp_dir",
|
||||
"storage.s3.endpoint",
|
||||
"storage.s3.region",
|
||||
"storage.s3.bucket",
|
||||
"storage.s3.access_key_id",
|
||||
"storage.s3.secret_access_key",
|
||||
"storage.s3.use_ssl",
|
||||
"storage.s3.path_style",
|
||||
"storage.presign.upload_expires",
|
||||
"storage.presign.download_expires",
|
||||
"logging.level",
|
||||
"logging.development",
|
||||
"logging.app_log.filename",
|
||||
"logging.app_log.max_size",
|
||||
"logging.app_log.max_backups",
|
||||
"logging.app_log.max_age",
|
||||
"logging.app_log.compress",
|
||||
"logging.access_log.filename",
|
||||
"logging.access_log.max_size",
|
||||
"logging.access_log.max_backups",
|
||||
"logging.access_log.max_age",
|
||||
"logging.access_log.compress",
|
||||
"queue.concurrency",
|
||||
"queue.retry_max",
|
||||
"queue.timeout",
|
||||
"jwt.secret_key",
|
||||
"jwt.token_duration",
|
||||
"jwt.access_token_ttl",
|
||||
"jwt.refresh_token_ttl",
|
||||
"middleware.enable_rate_limiter",
|
||||
"middleware.rate_limiter.max",
|
||||
"middleware.rate_limiter.expiration",
|
||||
"middleware.rate_limiter.storage",
|
||||
"sms.gateway_url",
|
||||
"sms.username",
|
||||
"sms.password",
|
||||
"sms.signature",
|
||||
"sms.timeout",
|
||||
"default_admin.username",
|
||||
"default_admin.password",
|
||||
"default_admin.phone",
|
||||
}
|
||||
|
||||
for _, key := range bindings {
|
||||
_ = v.BindEnv(key)
|
||||
}
|
||||
return absPath
|
||||
}
|
||||
|
||||
@@ -2,650 +2,219 @@ package config
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/break/junhong_cmp_fiber/pkg/constants"
|
||||
"github.com/spf13/viper"
|
||||
)
|
||||
|
||||
// TestLoad tests the config loading functionality
|
||||
func TestLoad(t *testing.T) {
|
||||
func TestLoad_EmbeddedConfig(t *testing.T) {
|
||||
clearEnvVars(t)
|
||||
setRequiredEnvVars(t)
|
||||
defer clearEnvVars(t)
|
||||
|
||||
cfg, err := Load()
|
||||
if err != nil {
|
||||
t.Fatalf("Load() 失败: %v", err)
|
||||
}
|
||||
|
||||
if cfg.Server.Address != ":3000" {
|
||||
t.Errorf("server.address 期望 :3000, 实际 %s", cfg.Server.Address)
|
||||
}
|
||||
if cfg.Server.ReadTimeout != 30*time.Second {
|
||||
t.Errorf("server.read_timeout 期望 30s, 实际 %v", cfg.Server.ReadTimeout)
|
||||
}
|
||||
if cfg.Logging.Level != "info" {
|
||||
t.Errorf("logging.level 期望 info, 实际 %s", cfg.Logging.Level)
|
||||
}
|
||||
}
|
||||
|
||||
func TestLoad_EnvOverride(t *testing.T) {
|
||||
clearEnvVars(t)
|
||||
setRequiredEnvVars(t)
|
||||
defer clearEnvVars(t)
|
||||
|
||||
os.Setenv("JUNHONG_SERVER_ADDRESS", ":8080")
|
||||
os.Setenv("JUNHONG_LOGGING_LEVEL", "debug")
|
||||
defer func() {
|
||||
os.Unsetenv("JUNHONG_SERVER_ADDRESS")
|
||||
os.Unsetenv("JUNHONG_LOGGING_LEVEL")
|
||||
}()
|
||||
|
||||
cfg, err := Load()
|
||||
if err != nil {
|
||||
t.Fatalf("Load() 失败: %v", err)
|
||||
}
|
||||
|
||||
if cfg.Server.Address != ":8080" {
|
||||
t.Errorf("server.address 期望 :8080, 实际 %s", cfg.Server.Address)
|
||||
}
|
||||
if cfg.Logging.Level != "debug" {
|
||||
t.Errorf("logging.level 期望 debug, 实际 %s", cfg.Logging.Level)
|
||||
}
|
||||
}
|
||||
|
||||
func TestLoad_MissingRequired(t *testing.T) {
|
||||
clearEnvVars(t)
|
||||
defer clearEnvVars(t)
|
||||
|
||||
_, err := Load()
|
||||
if err == nil {
|
||||
t.Fatal("Load() 缺少必填配置时应返回错误")
|
||||
}
|
||||
|
||||
expectedFields := []string{"database.host", "database.user", "database.password", "database.dbname", "redis.address", "jwt.secret_key"}
|
||||
for _, field := range expectedFields {
|
||||
if !containsString(err.Error(), field) {
|
||||
t.Errorf("错误信息应包含 %q, 实际: %s", field, err.Error())
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestLoad_PartialRequired(t *testing.T) {
|
||||
clearEnvVars(t)
|
||||
defer clearEnvVars(t)
|
||||
|
||||
os.Setenv("JUNHONG_DATABASE_HOST", "localhost")
|
||||
os.Setenv("JUNHONG_DATABASE_USER", "user")
|
||||
|
||||
_, err := Load()
|
||||
if err == nil {
|
||||
t.Fatal("Load() 部分必填配置缺失时应返回错误")
|
||||
}
|
||||
|
||||
if containsString(err.Error(), "database.host") {
|
||||
t.Error("database.host 已设置,不应在错误信息中")
|
||||
}
|
||||
if containsString(err.Error(), "database.user") {
|
||||
t.Error("database.user 已设置,不应在错误信息中")
|
||||
}
|
||||
if !containsString(err.Error(), "database.password") {
|
||||
t.Error("database.password 未设置,应在错误信息中")
|
||||
}
|
||||
}
|
||||
|
||||
func TestLoad_GlobalConfig(t *testing.T) {
|
||||
clearEnvVars(t)
|
||||
setRequiredEnvVars(t)
|
||||
defer clearEnvVars(t)
|
||||
|
||||
cfg, err := Load()
|
||||
if err != nil {
|
||||
t.Fatalf("Load() 失败: %v", err)
|
||||
}
|
||||
|
||||
globalCfg := Get()
|
||||
if globalCfg == nil {
|
||||
t.Fatal("Get() 返回 nil")
|
||||
}
|
||||
|
||||
if globalCfg.Server.Address != cfg.Server.Address {
|
||||
t.Errorf("全局配置与返回配置不一致")
|
||||
}
|
||||
}
|
||||
|
||||
func TestValidateRequired(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
setupEnv func()
|
||||
cleanupEnv func()
|
||||
createConfig func(t *testing.T) string
|
||||
wantErr bool
|
||||
validateFunc func(t *testing.T, cfg *Config)
|
||||
name string
|
||||
cfg *Config
|
||||
wantErr bool
|
||||
}{
|
||||
{
|
||||
name: "valid default config",
|
||||
setupEnv: func() {
|
||||
_ = os.Setenv(constants.EnvConfigPath, "")
|
||||
_ = os.Setenv(constants.EnvConfigEnv, "")
|
||||
},
|
||||
cleanupEnv: func() {
|
||||
_ = os.Unsetenv(constants.EnvConfigPath)
|
||||
_ = os.Unsetenv(constants.EnvConfigEnv)
|
||||
},
|
||||
createConfig: func(t *testing.T) string {
|
||||
t.Helper()
|
||||
tmpDir := t.TempDir()
|
||||
configFile := filepath.Join(tmpDir, "config.yaml")
|
||||
content := `
|
||||
server:
|
||||
address: ":3000"
|
||||
read_timeout: "10s"
|
||||
write_timeout: "10s"
|
||||
shutdown_timeout: "30s"
|
||||
prefork: false
|
||||
|
||||
redis:
|
||||
address: "localhost"
|
||||
port: 6379
|
||||
password: ""
|
||||
db: 0
|
||||
pool_size: 10
|
||||
min_idle_conns: 5
|
||||
dial_timeout: "5s"
|
||||
read_timeout: "3s"
|
||||
write_timeout: "3s"
|
||||
|
||||
logging:
|
||||
level: "info"
|
||||
development: false
|
||||
app_log:
|
||||
filename: "logs/app.log"
|
||||
max_size: 100
|
||||
max_backups: 30
|
||||
max_age: 30
|
||||
compress: true
|
||||
access_log:
|
||||
filename: "logs/access.log"
|
||||
max_size: 500
|
||||
max_backups: 90
|
||||
max_age: 90
|
||||
compress: true
|
||||
|
||||
middleware:
|
||||
enable_auth: true
|
||||
enable_rate_limiter: false
|
||||
rate_limiter:
|
||||
max: 100
|
||||
expiration: "1m"
|
||||
storage: "memory"
|
||||
`
|
||||
if err := os.WriteFile(configFile, []byte(content), 0644); err != nil {
|
||||
t.Fatalf("failed to create config file: %v", err)
|
||||
}
|
||||
// Set as default config path
|
||||
_ = os.Setenv(constants.EnvConfigPath, configFile)
|
||||
return configFile
|
||||
name: "all required set",
|
||||
cfg: &Config{
|
||||
Database: DatabaseConfig{
|
||||
Host: "localhost",
|
||||
User: "user",
|
||||
Password: "pass",
|
||||
DBName: "db",
|
||||
},
|
||||
Redis: RedisConfig{Address: "localhost"},
|
||||
JWT: JWTConfig{SecretKey: "12345678901234567890123456789012"},
|
||||
},
|
||||
wantErr: false,
|
||||
validateFunc: func(t *testing.T, cfg *Config) {
|
||||
if cfg.Server.Address != ":3000" {
|
||||
t.Errorf("expected server.address :3000, got %s", cfg.Server.Address)
|
||||
}
|
||||
if cfg.Server.ReadTimeout != 10*time.Second {
|
||||
t.Errorf("expected read_timeout 10s, got %v", cfg.Server.ReadTimeout)
|
||||
}
|
||||
if cfg.Redis.Address != "localhost" {
|
||||
t.Errorf("expected redis.address localhost, got %s", cfg.Redis.Address)
|
||||
}
|
||||
if cfg.Redis.Port != 6379 {
|
||||
t.Errorf("expected redis.port 6379, got %d", cfg.Redis.Port)
|
||||
}
|
||||
if cfg.Redis.PoolSize != 10 {
|
||||
t.Errorf("expected redis.pool_size 10, got %d", cfg.Redis.PoolSize)
|
||||
}
|
||||
if cfg.Logging.Level != "info" {
|
||||
t.Errorf("expected logging.level info, got %s", cfg.Logging.Level)
|
||||
}
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "environment-specific config (dev)",
|
||||
setupEnv: func() {
|
||||
_ = os.Setenv(constants.EnvConfigEnv, "dev")
|
||||
},
|
||||
cleanupEnv: func() {
|
||||
_ = os.Unsetenv(constants.EnvConfigEnv)
|
||||
_ = os.Unsetenv(constants.EnvConfigPath)
|
||||
},
|
||||
createConfig: func(t *testing.T) string {
|
||||
t.Helper()
|
||||
// Create configs directory in temp
|
||||
tmpDir := t.TempDir()
|
||||
configsDir := filepath.Join(tmpDir, "configs")
|
||||
if err := os.MkdirAll(configsDir, 0755); err != nil {
|
||||
t.Fatalf("failed to create configs dir: %v", err)
|
||||
}
|
||||
|
||||
// Create dev config
|
||||
devConfigFile := filepath.Join(configsDir, "config.dev.yaml")
|
||||
content := `
|
||||
server:
|
||||
address: ":8080"
|
||||
read_timeout: "15s"
|
||||
write_timeout: "15s"
|
||||
shutdown_timeout: "30s"
|
||||
prefork: false
|
||||
|
||||
redis:
|
||||
address: "localhost"
|
||||
port: 6379
|
||||
password: ""
|
||||
db: 1
|
||||
pool_size: 5
|
||||
min_idle_conns: 2
|
||||
dial_timeout: "5s"
|
||||
read_timeout: "3s"
|
||||
write_timeout: "3s"
|
||||
|
||||
logging:
|
||||
level: "debug"
|
||||
development: true
|
||||
app_log:
|
||||
filename: "logs/app.log"
|
||||
max_size: 50
|
||||
max_backups: 10
|
||||
max_age: 7
|
||||
compress: false
|
||||
access_log:
|
||||
filename: "logs/access.log"
|
||||
max_size: 100
|
||||
max_backups: 30
|
||||
max_age: 30
|
||||
compress: false
|
||||
|
||||
middleware:
|
||||
enable_rate_limiter: false
|
||||
rate_limiter:
|
||||
max: 50
|
||||
expiration: "1m"
|
||||
storage: "memory"
|
||||
`
|
||||
if err := os.WriteFile(devConfigFile, []byte(content), 0644); err != nil {
|
||||
t.Fatalf("failed to create dev config file: %v", err)
|
||||
}
|
||||
|
||||
// Change to tmpDir so relative path works
|
||||
originalWd, _ := os.Getwd()
|
||||
_ = os.Chdir(tmpDir)
|
||||
t.Cleanup(func() { _ = os.Chdir(originalWd) })
|
||||
|
||||
return devConfigFile
|
||||
},
|
||||
wantErr: false,
|
||||
validateFunc: func(t *testing.T, cfg *Config) {
|
||||
if cfg.Server.Address != ":8080" {
|
||||
t.Errorf("expected server.address :8080, got %s", cfg.Server.Address)
|
||||
}
|
||||
if cfg.Redis.DB != 1 {
|
||||
t.Errorf("expected redis.db 1, got %d", cfg.Redis.DB)
|
||||
}
|
||||
if cfg.Logging.Level != "debug" {
|
||||
t.Errorf("expected logging.level debug, got %s", cfg.Logging.Level)
|
||||
}
|
||||
name: "missing database host",
|
||||
cfg: &Config{
|
||||
Database: DatabaseConfig{
|
||||
User: "user",
|
||||
Password: "pass",
|
||||
DBName: "db",
|
||||
},
|
||||
Redis: RedisConfig{Address: "localhost"},
|
||||
JWT: JWTConfig{SecretKey: "12345678901234567890123456789012"},
|
||||
},
|
||||
wantErr: true,
|
||||
},
|
||||
{
|
||||
name: "invalid YAML syntax",
|
||||
setupEnv: func() {
|
||||
_ = os.Setenv(constants.EnvConfigPath, "")
|
||||
_ = os.Setenv(constants.EnvConfigEnv, "")
|
||||
name: "missing redis address",
|
||||
cfg: &Config{
|
||||
Database: DatabaseConfig{
|
||||
Host: "localhost",
|
||||
User: "user",
|
||||
Password: "pass",
|
||||
DBName: "db",
|
||||
},
|
||||
Redis: RedisConfig{},
|
||||
JWT: JWTConfig{SecretKey: "12345678901234567890123456789012"},
|
||||
},
|
||||
cleanupEnv: func() {
|
||||
_ = os.Unsetenv(constants.EnvConfigPath)
|
||||
_ = os.Unsetenv(constants.EnvConfigEnv)
|
||||
},
|
||||
createConfig: func(t *testing.T) string {
|
||||
t.Helper()
|
||||
tmpDir := t.TempDir()
|
||||
configFile := filepath.Join(tmpDir, "config.yaml")
|
||||
content := `
|
||||
server:
|
||||
address: ":3000"
|
||||
invalid yaml syntax here!!!
|
||||
`
|
||||
if err := os.WriteFile(configFile, []byte(content), 0644); err != nil {
|
||||
t.Fatalf("failed to create config file: %v", err)
|
||||
}
|
||||
_ = os.Setenv(constants.EnvConfigPath, configFile)
|
||||
return configFile
|
||||
},
|
||||
wantErr: true,
|
||||
validateFunc: nil,
|
||||
wantErr: true,
|
||||
},
|
||||
{
|
||||
name: "validation error - invalid server address",
|
||||
setupEnv: func() {
|
||||
_ = os.Setenv(constants.EnvConfigPath, "")
|
||||
name: "missing jwt secret",
|
||||
cfg: &Config{
|
||||
Database: DatabaseConfig{
|
||||
Host: "localhost",
|
||||
User: "user",
|
||||
Password: "pass",
|
||||
DBName: "db",
|
||||
},
|
||||
Redis: RedisConfig{Address: "localhost"},
|
||||
JWT: JWTConfig{},
|
||||
},
|
||||
cleanupEnv: func() {
|
||||
_ = os.Unsetenv(constants.EnvConfigPath)
|
||||
},
|
||||
createConfig: func(t *testing.T) string {
|
||||
t.Helper()
|
||||
tmpDir := t.TempDir()
|
||||
configFile := filepath.Join(tmpDir, "config.yaml")
|
||||
content := `
|
||||
server:
|
||||
address: ""
|
||||
read_timeout: "10s"
|
||||
write_timeout: "10s"
|
||||
shutdown_timeout: "30s"
|
||||
|
||||
redis:
|
||||
address: "localhost"
|
||||
port: 6379
|
||||
db: 0
|
||||
pool_size: 10
|
||||
min_idle_conns: 5
|
||||
|
||||
logging:
|
||||
level: "info"
|
||||
app_log:
|
||||
filename: "logs/app.log"
|
||||
max_size: 100
|
||||
max_backups: 30
|
||||
max_age: 30
|
||||
compress: true
|
||||
access_log:
|
||||
filename: "logs/access.log"
|
||||
max_size: 500
|
||||
max_backups: 90
|
||||
max_age: 90
|
||||
compress: true
|
||||
|
||||
middleware:
|
||||
enable_auth: true
|
||||
rate_limiter:
|
||||
max: 100
|
||||
expiration: "1m"
|
||||
storage: "memory"
|
||||
`
|
||||
if err := os.WriteFile(configFile, []byte(content), 0644); err != nil {
|
||||
t.Fatalf("failed to create config file: %v", err)
|
||||
}
|
||||
_ = os.Setenv(constants.EnvConfigPath, configFile)
|
||||
return configFile
|
||||
},
|
||||
wantErr: true,
|
||||
validateFunc: nil,
|
||||
},
|
||||
{
|
||||
name: "validation error - timeout out of range",
|
||||
setupEnv: func() {
|
||||
_ = os.Setenv(constants.EnvConfigPath, "")
|
||||
},
|
||||
cleanupEnv: func() {
|
||||
_ = os.Unsetenv(constants.EnvConfigPath)
|
||||
},
|
||||
createConfig: func(t *testing.T) string {
|
||||
t.Helper()
|
||||
tmpDir := t.TempDir()
|
||||
configFile := filepath.Join(tmpDir, "config.yaml")
|
||||
content := `
|
||||
server:
|
||||
address: ":3000"
|
||||
read_timeout: "1s"
|
||||
write_timeout: "10s"
|
||||
shutdown_timeout: "30s"
|
||||
|
||||
redis:
|
||||
address: "localhost"
|
||||
port: 6379
|
||||
db: 0
|
||||
pool_size: 10
|
||||
min_idle_conns: 5
|
||||
|
||||
logging:
|
||||
level: "info"
|
||||
app_log:
|
||||
filename: "logs/app.log"
|
||||
max_size: 100
|
||||
max_backups: 30
|
||||
max_age: 30
|
||||
compress: true
|
||||
access_log:
|
||||
filename: "logs/access.log"
|
||||
max_size: 500
|
||||
max_backups: 90
|
||||
max_age: 90
|
||||
compress: true
|
||||
|
||||
middleware:
|
||||
enable_auth: true
|
||||
rate_limiter:
|
||||
max: 100
|
||||
expiration: "1m"
|
||||
storage: "memory"
|
||||
`
|
||||
if err := os.WriteFile(configFile, []byte(content), 0644); err != nil {
|
||||
t.Fatalf("failed to create config file: %v", err)
|
||||
}
|
||||
_ = os.Setenv(constants.EnvConfigPath, configFile)
|
||||
return configFile
|
||||
},
|
||||
wantErr: true,
|
||||
validateFunc: nil,
|
||||
},
|
||||
{
|
||||
name: "validation error - invalid redis port",
|
||||
setupEnv: func() {
|
||||
_ = os.Setenv(constants.EnvConfigPath, "")
|
||||
},
|
||||
cleanupEnv: func() {
|
||||
_ = os.Unsetenv(constants.EnvConfigPath)
|
||||
},
|
||||
createConfig: func(t *testing.T) string {
|
||||
t.Helper()
|
||||
tmpDir := t.TempDir()
|
||||
configFile := filepath.Join(tmpDir, "config.yaml")
|
||||
content := `
|
||||
server:
|
||||
address: ":3000"
|
||||
read_timeout: "10s"
|
||||
write_timeout: "10s"
|
||||
shutdown_timeout: "30s"
|
||||
|
||||
redis:
|
||||
address: "localhost"
|
||||
port: 99999
|
||||
db: 0
|
||||
pool_size: 10
|
||||
min_idle_conns: 5
|
||||
|
||||
logging:
|
||||
level: "info"
|
||||
app_log:
|
||||
filename: "logs/app.log"
|
||||
max_size: 100
|
||||
max_backups: 30
|
||||
max_age: 30
|
||||
compress: true
|
||||
access_log:
|
||||
filename: "logs/access.log"
|
||||
max_size: 500
|
||||
max_backups: 90
|
||||
max_age: 90
|
||||
compress: true
|
||||
|
||||
middleware:
|
||||
enable_auth: true
|
||||
rate_limiter:
|
||||
max: 100
|
||||
expiration: "1m"
|
||||
storage: "memory"
|
||||
`
|
||||
if err := os.WriteFile(configFile, []byte(content), 0644); err != nil {
|
||||
t.Fatalf("failed to create config file: %v", err)
|
||||
}
|
||||
_ = os.Setenv(constants.EnvConfigPath, configFile)
|
||||
return configFile
|
||||
},
|
||||
wantErr: true,
|
||||
validateFunc: nil,
|
||||
wantErr: true,
|
||||
},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
// Reset viper for each test
|
||||
viper.Reset()
|
||||
|
||||
// Setup environment
|
||||
if tt.setupEnv != nil {
|
||||
tt.setupEnv()
|
||||
}
|
||||
|
||||
// Create config file
|
||||
if tt.createConfig != nil {
|
||||
tt.createConfig(t)
|
||||
}
|
||||
|
||||
// Cleanup after test
|
||||
if tt.cleanupEnv != nil {
|
||||
defer tt.cleanupEnv()
|
||||
}
|
||||
|
||||
// Load config
|
||||
cfg, err := Load()
|
||||
|
||||
// Check error expectation
|
||||
err := tt.cfg.ValidateRequired()
|
||||
if (err != nil) != tt.wantErr {
|
||||
t.Errorf("Load() error = %v, wantErr %v", err, tt.wantErr)
|
||||
return
|
||||
}
|
||||
|
||||
// Validate config if no error expected
|
||||
if !tt.wantErr && tt.validateFunc != nil {
|
||||
tt.validateFunc(t, cfg)
|
||||
t.Errorf("ValidateRequired() error = %v, wantErr %v", err, tt.wantErr)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestReload tests the config reload functionality
|
||||
func TestReload(t *testing.T) {
|
||||
// Reset viper
|
||||
viper.Reset()
|
||||
func setRequiredEnvVars(t *testing.T) {
|
||||
t.Helper()
|
||||
os.Setenv("JUNHONG_DATABASE_HOST", "localhost")
|
||||
os.Setenv("JUNHONG_DATABASE_USER", "testuser")
|
||||
os.Setenv("JUNHONG_DATABASE_PASSWORD", "testpass")
|
||||
os.Setenv("JUNHONG_DATABASE_DBNAME", "testdb")
|
||||
os.Setenv("JUNHONG_REDIS_ADDRESS", "localhost")
|
||||
os.Setenv("JUNHONG_JWT_SECRET_KEY", "12345678901234567890123456789012")
|
||||
}
|
||||
|
||||
// Create temp config file
|
||||
tmpDir := t.TempDir()
|
||||
configFile := filepath.Join(tmpDir, "config.yaml")
|
||||
|
||||
// Initial config
|
||||
initialContent := `
|
||||
server:
|
||||
address: ":3000"
|
||||
read_timeout: "10s"
|
||||
write_timeout: "10s"
|
||||
shutdown_timeout: "30s"
|
||||
prefork: false
|
||||
|
||||
redis:
|
||||
address: "localhost"
|
||||
port: 6379
|
||||
password: ""
|
||||
db: 0
|
||||
pool_size: 10
|
||||
min_idle_conns: 5
|
||||
dial_timeout: "5s"
|
||||
read_timeout: "3s"
|
||||
write_timeout: "3s"
|
||||
|
||||
logging:
|
||||
level: "info"
|
||||
development: false
|
||||
app_log:
|
||||
filename: "logs/app.log"
|
||||
max_size: 100
|
||||
max_backups: 30
|
||||
max_age: 30
|
||||
compress: true
|
||||
access_log:
|
||||
filename: "logs/access.log"
|
||||
max_size: 500
|
||||
max_backups: 90
|
||||
max_age: 90
|
||||
compress: true
|
||||
|
||||
middleware:
|
||||
enable_auth: true
|
||||
enable_rate_limiter: false
|
||||
rate_limiter:
|
||||
max: 100
|
||||
expiration: "1m"
|
||||
storage: "memory"
|
||||
`
|
||||
if err := os.WriteFile(configFile, []byte(initialContent), 0644); err != nil {
|
||||
t.Fatalf("failed to create config file: %v", err)
|
||||
func clearEnvVars(t *testing.T) {
|
||||
t.Helper()
|
||||
envVars := []string{
|
||||
"JUNHONG_DATABASE_HOST",
|
||||
"JUNHONG_DATABASE_PORT",
|
||||
"JUNHONG_DATABASE_USER",
|
||||
"JUNHONG_DATABASE_PASSWORD",
|
||||
"JUNHONG_DATABASE_DBNAME",
|
||||
"JUNHONG_REDIS_ADDRESS",
|
||||
"JUNHONG_REDIS_PORT",
|
||||
"JUNHONG_REDIS_PASSWORD",
|
||||
"JUNHONG_JWT_SECRET_KEY",
|
||||
"JUNHONG_SERVER_ADDRESS",
|
||||
"JUNHONG_LOGGING_LEVEL",
|
||||
}
|
||||
|
||||
// Set config path
|
||||
_ = os.Setenv(constants.EnvConfigPath, configFile)
|
||||
defer func() { _ = os.Unsetenv(constants.EnvConfigPath) }()
|
||||
|
||||
// Load initial config
|
||||
cfg, err := Load()
|
||||
if err != nil {
|
||||
t.Fatalf("failed to load initial config: %v", err)
|
||||
}
|
||||
|
||||
// Verify initial values
|
||||
if cfg.Logging.Level != "info" {
|
||||
t.Errorf("expected initial logging.level info, got %s", cfg.Logging.Level)
|
||||
}
|
||||
if cfg.Server.Address != ":3000" {
|
||||
t.Errorf("expected initial server.address :3000, got %s", cfg.Server.Address)
|
||||
}
|
||||
|
||||
// Modify config file
|
||||
updatedContent := `
|
||||
server:
|
||||
address: ":8080"
|
||||
read_timeout: "15s"
|
||||
write_timeout: "15s"
|
||||
shutdown_timeout: "30s"
|
||||
prefork: false
|
||||
|
||||
redis:
|
||||
address: "localhost"
|
||||
port: 6379
|
||||
password: ""
|
||||
db: 0
|
||||
pool_size: 20
|
||||
min_idle_conns: 10
|
||||
dial_timeout: "5s"
|
||||
read_timeout: "3s"
|
||||
write_timeout: "3s"
|
||||
|
||||
logging:
|
||||
level: "debug"
|
||||
development: true
|
||||
app_log:
|
||||
filename: "logs/app.log"
|
||||
max_size: 100
|
||||
max_backups: 30
|
||||
max_age: 30
|
||||
compress: true
|
||||
access_log:
|
||||
filename: "logs/access.log"
|
||||
max_size: 500
|
||||
max_backups: 90
|
||||
max_age: 90
|
||||
compress: true
|
||||
|
||||
middleware:
|
||||
enable_auth: false
|
||||
enable_rate_limiter: true
|
||||
rate_limiter:
|
||||
max: 200
|
||||
expiration: "2m"
|
||||
storage: "redis"
|
||||
`
|
||||
if err := os.WriteFile(configFile, []byte(updatedContent), 0644); err != nil {
|
||||
t.Fatalf("failed to update config file: %v", err)
|
||||
}
|
||||
|
||||
// Reload config
|
||||
newCfg, err := Reload()
|
||||
if err != nil {
|
||||
t.Fatalf("failed to reload config: %v", err)
|
||||
}
|
||||
|
||||
// Verify updated values
|
||||
if newCfg.Logging.Level != "debug" {
|
||||
t.Errorf("expected updated logging.level debug, got %s", newCfg.Logging.Level)
|
||||
}
|
||||
if newCfg.Server.Address != ":8080" {
|
||||
t.Errorf("expected updated server.address :8080, got %s", newCfg.Server.Address)
|
||||
}
|
||||
if newCfg.Redis.PoolSize != 20 {
|
||||
t.Errorf("expected updated redis.pool_size 20, got %d", newCfg.Redis.PoolSize)
|
||||
}
|
||||
if newCfg.Middleware.EnableRateLimiter != true {
|
||||
t.Errorf("expected updated enable_rate_limiter true, got %v", newCfg.Middleware.EnableRateLimiter)
|
||||
}
|
||||
|
||||
// Verify global config was updated
|
||||
globalCfg := Get()
|
||||
if globalCfg.Logging.Level != "debug" {
|
||||
t.Errorf("expected global config updated, got logging.level %s", globalCfg.Logging.Level)
|
||||
for _, v := range envVars {
|
||||
os.Unsetenv(v)
|
||||
}
|
||||
}
|
||||
|
||||
// TestGetConfigPath tests the GetConfigPath function
|
||||
func TestGetConfigPath(t *testing.T) {
|
||||
// Reset viper
|
||||
viper.Reset()
|
||||
|
||||
// Create temp config file
|
||||
tmpDir := t.TempDir()
|
||||
configFile := filepath.Join(tmpDir, "config.yaml")
|
||||
|
||||
content := `
|
||||
server:
|
||||
address: ":3000"
|
||||
read_timeout: "10s"
|
||||
write_timeout: "10s"
|
||||
shutdown_timeout: "30s"
|
||||
|
||||
redis:
|
||||
address: "localhost"
|
||||
port: 6379
|
||||
db: 0
|
||||
pool_size: 10
|
||||
min_idle_conns: 5
|
||||
|
||||
logging:
|
||||
level: "info"
|
||||
app_log:
|
||||
filename: "logs/app.log"
|
||||
max_size: 100
|
||||
max_backups: 30
|
||||
max_age: 30
|
||||
compress: true
|
||||
access_log:
|
||||
filename: "logs/access.log"
|
||||
max_size: 500
|
||||
max_backups: 90
|
||||
max_age: 90
|
||||
compress: true
|
||||
|
||||
middleware:
|
||||
enable_auth: true
|
||||
rate_limiter:
|
||||
max: 100
|
||||
expiration: "1m"
|
||||
storage: "memory"
|
||||
`
|
||||
if err := os.WriteFile(configFile, []byte(content), 0644); err != nil {
|
||||
t.Fatalf("failed to create config file: %v", err)
|
||||
}
|
||||
|
||||
_ = os.Setenv(constants.EnvConfigPath, configFile)
|
||||
defer func() { _ = os.Unsetenv(constants.EnvConfigPath) }()
|
||||
|
||||
// Load config
|
||||
_, err := Load()
|
||||
if err != nil {
|
||||
t.Fatalf("failed to load config: %v", err)
|
||||
}
|
||||
|
||||
// Get config path
|
||||
path := GetConfigPath()
|
||||
if path == "" {
|
||||
t.Error("expected non-empty config path")
|
||||
}
|
||||
|
||||
// Verify it's an absolute path
|
||||
if !filepath.IsAbs(path) {
|
||||
t.Errorf("expected absolute path, got %s", path)
|
||||
}
|
||||
func containsString(s, substr string) bool {
|
||||
return len(s) >= len(substr) && (s == substr || len(s) > 0 && (s[:len(substr)] == substr || containsString(s[1:], substr)))
|
||||
}
|
||||
|
||||
@@ -1,43 +0,0 @@
|
||||
package config
|
||||
|
||||
import (
|
||||
"context"
|
||||
|
||||
"github.com/fsnotify/fsnotify"
|
||||
"github.com/spf13/viper"
|
||||
"go.uber.org/zap"
|
||||
)
|
||||
|
||||
// Watch 监听配置文件变化
|
||||
// 运行直到上下文被取消
|
||||
func Watch(ctx context.Context, logger *zap.Logger) {
|
||||
viper.WatchConfig()
|
||||
viper.OnConfigChange(func(e fsnotify.Event) {
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return // 如果上下文被取消则停止处理
|
||||
default:
|
||||
logger.Info("配置文件已更改", zap.String("file", e.Name))
|
||||
|
||||
// 尝试重新加载
|
||||
newConfig, err := Reload()
|
||||
if err != nil {
|
||||
logger.Error("重新加载配置失败,保留先前配置",
|
||||
zap.Error(err),
|
||||
zap.String("file", e.Name),
|
||||
)
|
||||
return
|
||||
}
|
||||
|
||||
logger.Info("配置重新加载成功",
|
||||
zap.String("file", e.Name),
|
||||
zap.String("server_address", newConfig.Server.Address),
|
||||
zap.String("log_level", newConfig.Logging.Level),
|
||||
)
|
||||
}
|
||||
})
|
||||
|
||||
// 阻塞直到上下文被取消
|
||||
<-ctx.Done()
|
||||
logger.Info("配置监听器已停止")
|
||||
}
|
||||
@@ -1,422 +0,0 @@
|
||||
package config
|
||||
|
||||
import (
|
||||
"context"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/break/junhong_cmp_fiber/pkg/constants"
|
||||
"github.com/spf13/viper"
|
||||
"go.uber.org/zap"
|
||||
"go.uber.org/zap/zaptest"
|
||||
)
|
||||
|
||||
// TestWatch tests the config hot reload watcher
|
||||
func TestWatch(t *testing.T) {
|
||||
// Reset viper
|
||||
viper.Reset()
|
||||
|
||||
// Create temp config file
|
||||
tmpDir := t.TempDir()
|
||||
configFile := filepath.Join(tmpDir, "config.yaml")
|
||||
|
||||
// Initial config
|
||||
initialContent := `
|
||||
server:
|
||||
address: ":3000"
|
||||
read_timeout: "10s"
|
||||
write_timeout: "10s"
|
||||
shutdown_timeout: "30s"
|
||||
prefork: false
|
||||
|
||||
redis:
|
||||
address: "localhost"
|
||||
port: 6379
|
||||
password: ""
|
||||
db: 0
|
||||
pool_size: 10
|
||||
min_idle_conns: 5
|
||||
dial_timeout: "5s"
|
||||
read_timeout: "3s"
|
||||
write_timeout: "3s"
|
||||
|
||||
logging:
|
||||
level: "info"
|
||||
development: false
|
||||
app_log:
|
||||
filename: "logs/app.log"
|
||||
max_size: 100
|
||||
max_backups: 30
|
||||
max_age: 30
|
||||
compress: true
|
||||
access_log:
|
||||
filename: "logs/access.log"
|
||||
max_size: 500
|
||||
max_backups: 90
|
||||
max_age: 90
|
||||
compress: true
|
||||
|
||||
middleware:
|
||||
enable_auth: true
|
||||
enable_rate_limiter: false
|
||||
rate_limiter:
|
||||
max: 100
|
||||
expiration: "1m"
|
||||
storage: "memory"
|
||||
`
|
||||
if err := os.WriteFile(configFile, []byte(initialContent), 0644); err != nil {
|
||||
t.Fatalf("failed to create config file: %v", err)
|
||||
}
|
||||
|
||||
// Set config path
|
||||
_ = os.Setenv(constants.EnvConfigPath, configFile)
|
||||
defer func() { _ = os.Unsetenv(constants.EnvConfigPath) }()
|
||||
|
||||
// Load initial config
|
||||
cfg, err := Load()
|
||||
if err != nil {
|
||||
t.Fatalf("failed to load initial config: %v", err)
|
||||
}
|
||||
|
||||
// Verify initial values
|
||||
if cfg.Logging.Level != "info" {
|
||||
t.Fatalf("expected initial logging.level info, got %s", cfg.Logging.Level)
|
||||
}
|
||||
|
||||
// Create logger for testing
|
||||
logger := zaptest.NewLogger(t)
|
||||
|
||||
// Start watcher in goroutine with context
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
defer cancel()
|
||||
|
||||
go Watch(ctx, logger)
|
||||
|
||||
// Give watcher time to initialize
|
||||
time.Sleep(100 * time.Millisecond)
|
||||
|
||||
// Modify config file to trigger hot reload
|
||||
updatedContent := `
|
||||
server:
|
||||
address: ":8080"
|
||||
read_timeout: "15s"
|
||||
write_timeout: "15s"
|
||||
shutdown_timeout: "30s"
|
||||
prefork: false
|
||||
|
||||
redis:
|
||||
address: "localhost"
|
||||
port: 6379
|
||||
password: ""
|
||||
db: 0
|
||||
pool_size: 20
|
||||
min_idle_conns: 10
|
||||
dial_timeout: "5s"
|
||||
read_timeout: "3s"
|
||||
write_timeout: "3s"
|
||||
|
||||
logging:
|
||||
level: "debug"
|
||||
development: true
|
||||
app_log:
|
||||
filename: "logs/app.log"
|
||||
max_size: 100
|
||||
max_backups: 30
|
||||
max_age: 30
|
||||
compress: true
|
||||
access_log:
|
||||
filename: "logs/access.log"
|
||||
max_size: 500
|
||||
max_backups: 90
|
||||
max_age: 90
|
||||
compress: true
|
||||
|
||||
middleware:
|
||||
enable_auth: false
|
||||
enable_rate_limiter: true
|
||||
rate_limiter:
|
||||
max: 200
|
||||
expiration: "2m"
|
||||
storage: "redis"
|
||||
`
|
||||
if err := os.WriteFile(configFile, []byte(updatedContent), 0644); err != nil {
|
||||
t.Fatalf("failed to update config file: %v", err)
|
||||
}
|
||||
|
||||
// Wait for watcher to detect and process changes (spec requires detection within 5 seconds)
|
||||
// We use a more aggressive timeout for testing
|
||||
time.Sleep(2 * time.Second)
|
||||
|
||||
// Verify config was reloaded
|
||||
reloadedCfg := Get()
|
||||
if reloadedCfg.Logging.Level != "debug" {
|
||||
t.Errorf("expected config hot reload, got logging.level %s instead of debug", reloadedCfg.Logging.Level)
|
||||
}
|
||||
if reloadedCfg.Server.Address != ":8080" {
|
||||
t.Errorf("expected config hot reload, got server.address %s instead of :8080", reloadedCfg.Server.Address)
|
||||
}
|
||||
if reloadedCfg.Redis.PoolSize != 20 {
|
||||
t.Errorf("expected config hot reload, got redis.pool_size %d instead of 20", reloadedCfg.Redis.PoolSize)
|
||||
}
|
||||
|
||||
// Cancel context to stop watcher
|
||||
cancel()
|
||||
|
||||
// Give watcher time to shut down gracefully
|
||||
time.Sleep(100 * time.Millisecond)
|
||||
}
|
||||
|
||||
// TestWatch_InvalidConfigRejected tests that invalid config changes are rejected
|
||||
func TestWatch_InvalidConfigRejected(t *testing.T) {
|
||||
// Reset viper
|
||||
viper.Reset()
|
||||
|
||||
// Create temp config file
|
||||
tmpDir := t.TempDir()
|
||||
configFile := filepath.Join(tmpDir, "config.yaml")
|
||||
|
||||
// Initial valid config
|
||||
validContent := `
|
||||
server:
|
||||
address: ":3000"
|
||||
read_timeout: "10s"
|
||||
write_timeout: "10s"
|
||||
shutdown_timeout: "30s"
|
||||
prefork: false
|
||||
|
||||
redis:
|
||||
address: "localhost"
|
||||
port: 6379
|
||||
password: ""
|
||||
db: 0
|
||||
pool_size: 10
|
||||
min_idle_conns: 5
|
||||
dial_timeout: "5s"
|
||||
read_timeout: "3s"
|
||||
write_timeout: "3s"
|
||||
|
||||
logging:
|
||||
level: "info"
|
||||
development: false
|
||||
app_log:
|
||||
filename: "logs/app.log"
|
||||
max_size: 100
|
||||
max_backups: 30
|
||||
max_age: 30
|
||||
compress: true
|
||||
access_log:
|
||||
filename: "logs/access.log"
|
||||
max_size: 500
|
||||
max_backups: 90
|
||||
max_age: 90
|
||||
compress: true
|
||||
|
||||
middleware:
|
||||
enable_auth: true
|
||||
enable_rate_limiter: false
|
||||
rate_limiter:
|
||||
max: 100
|
||||
expiration: "1m"
|
||||
storage: "memory"
|
||||
`
|
||||
if err := os.WriteFile(configFile, []byte(validContent), 0644); err != nil {
|
||||
t.Fatalf("failed to create config file: %v", err)
|
||||
}
|
||||
|
||||
// Set config path
|
||||
_ = os.Setenv(constants.EnvConfigPath, configFile)
|
||||
defer func() { _ = os.Unsetenv(constants.EnvConfigPath) }()
|
||||
|
||||
// Load initial config
|
||||
cfg, err := Load()
|
||||
if err != nil {
|
||||
t.Fatalf("failed to load initial config: %v", err)
|
||||
}
|
||||
|
||||
initialLevel := cfg.Logging.Level
|
||||
if initialLevel != "info" {
|
||||
t.Fatalf("expected initial logging.level info, got %s", initialLevel)
|
||||
}
|
||||
|
||||
// Create logger for testing
|
||||
logger := zaptest.NewLogger(t)
|
||||
|
||||
// Start watcher
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
defer cancel()
|
||||
|
||||
go Watch(ctx, logger)
|
||||
|
||||
// Give watcher time to initialize
|
||||
time.Sleep(100 * time.Millisecond)
|
||||
|
||||
// Write INVALID config (malformed YAML)
|
||||
invalidContent := `
|
||||
server:
|
||||
address: ":3000"
|
||||
invalid yaml syntax here!!!
|
||||
`
|
||||
if err := os.WriteFile(configFile, []byte(invalidContent), 0644); err != nil {
|
||||
t.Fatalf("failed to write invalid config: %v", err)
|
||||
}
|
||||
|
||||
// Wait for watcher to detect changes
|
||||
time.Sleep(2 * time.Second)
|
||||
|
||||
// Verify config was NOT changed (should keep previous valid config)
|
||||
currentCfg := Get()
|
||||
if currentCfg.Logging.Level != initialLevel {
|
||||
t.Errorf("expected config to remain unchanged after invalid update, got logging.level %s instead of %s", currentCfg.Logging.Level, initialLevel)
|
||||
}
|
||||
|
||||
// Restore valid config
|
||||
if err := os.WriteFile(configFile, []byte(validContent), 0644); err != nil {
|
||||
t.Fatalf("failed to restore valid config: %v", err)
|
||||
}
|
||||
|
||||
time.Sleep(500 * time.Millisecond)
|
||||
|
||||
// Now write config with validation error (timeout out of range)
|
||||
invalidValidationContent := `
|
||||
server:
|
||||
address: ":3000"
|
||||
read_timeout: "1s"
|
||||
write_timeout: "10s"
|
||||
shutdown_timeout: "30s"
|
||||
|
||||
redis:
|
||||
address: "localhost"
|
||||
port: 6379
|
||||
db: 0
|
||||
pool_size: 10
|
||||
min_idle_conns: 5
|
||||
|
||||
logging:
|
||||
level: "debug"
|
||||
app_log:
|
||||
filename: "logs/app.log"
|
||||
max_size: 100
|
||||
max_backups: 30
|
||||
max_age: 30
|
||||
compress: true
|
||||
access_log:
|
||||
filename: "logs/access.log"
|
||||
max_size: 500
|
||||
max_backups: 90
|
||||
max_age: 90
|
||||
compress: true
|
||||
|
||||
middleware:
|
||||
enable_auth: true
|
||||
rate_limiter:
|
||||
max: 100
|
||||
expiration: "1m"
|
||||
storage: "memory"
|
||||
`
|
||||
if err := os.WriteFile(configFile, []byte(invalidValidationContent), 0644); err != nil {
|
||||
t.Fatalf("failed to write config with validation error: %v", err)
|
||||
}
|
||||
|
||||
// Wait for watcher to detect changes
|
||||
time.Sleep(2 * time.Second)
|
||||
|
||||
// Verify config was NOT changed (validation should have failed)
|
||||
finalCfg := Get()
|
||||
if finalCfg.Logging.Level != initialLevel {
|
||||
t.Errorf("expected config to remain unchanged after validation error, got logging.level %s instead of %s", finalCfg.Logging.Level, initialLevel)
|
||||
}
|
||||
if finalCfg.Server.ReadTimeout == 1*time.Second {
|
||||
t.Error("expected config to remain unchanged, but read_timeout was updated to invalid value")
|
||||
}
|
||||
|
||||
// Cancel context
|
||||
cancel()
|
||||
time.Sleep(100 * time.Millisecond)
|
||||
}
|
||||
|
||||
// TestWatch_ContextCancellation tests graceful shutdown on context cancellation
|
||||
func TestWatch_ContextCancellation(t *testing.T) {
|
||||
// Reset viper
|
||||
viper.Reset()
|
||||
|
||||
// Create temp config file
|
||||
tmpDir := t.TempDir()
|
||||
configFile := filepath.Join(tmpDir, "config.yaml")
|
||||
|
||||
content := `
|
||||
server:
|
||||
address: ":3000"
|
||||
read_timeout: "10s"
|
||||
write_timeout: "10s"
|
||||
shutdown_timeout: "30s"
|
||||
|
||||
redis:
|
||||
address: "localhost"
|
||||
port: 6379
|
||||
db: 0
|
||||
pool_size: 10
|
||||
min_idle_conns: 5
|
||||
|
||||
logging:
|
||||
level: "info"
|
||||
app_log:
|
||||
filename: "logs/app.log"
|
||||
max_size: 100
|
||||
max_backups: 30
|
||||
max_age: 30
|
||||
compress: true
|
||||
access_log:
|
||||
filename: "logs/access.log"
|
||||
max_size: 500
|
||||
max_backups: 90
|
||||
max_age: 90
|
||||
compress: true
|
||||
|
||||
middleware:
|
||||
enable_auth: true
|
||||
rate_limiter:
|
||||
max: 100
|
||||
expiration: "1m"
|
||||
storage: "memory"
|
||||
`
|
||||
if err := os.WriteFile(configFile, []byte(content), 0644); err != nil {
|
||||
t.Fatalf("failed to create config file: %v", err)
|
||||
}
|
||||
|
||||
_ = os.Setenv(constants.EnvConfigPath, configFile)
|
||||
defer func() { _ = os.Unsetenv(constants.EnvConfigPath) }()
|
||||
|
||||
// Load config
|
||||
_, err := Load()
|
||||
if err != nil {
|
||||
t.Fatalf("failed to load config: %v", err)
|
||||
}
|
||||
|
||||
// Create logger
|
||||
logger := zap.NewNop() // Use no-op logger for this test
|
||||
|
||||
// Start watcher with context
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
|
||||
done := make(chan bool)
|
||||
go func() {
|
||||
Watch(ctx, logger)
|
||||
done <- true
|
||||
}()
|
||||
|
||||
// Give watcher time to start
|
||||
time.Sleep(100 * time.Millisecond)
|
||||
|
||||
// Cancel context (simulate graceful shutdown)
|
||||
cancel()
|
||||
|
||||
// Wait for watcher to stop (should happen quickly)
|
||||
select {
|
||||
case <-done:
|
||||
// Watcher stopped successfully
|
||||
case <-time.After(2 * time.Second):
|
||||
t.Error("watcher did not stop within timeout after context cancellation")
|
||||
}
|
||||
}
|
||||
@@ -68,6 +68,14 @@ const (
|
||||
CodeCannotAllocateToSelf = 1075 // 不能分配给自己
|
||||
CodeCannotRecallFromSelf = 1076 // 不能从自己回收
|
||||
|
||||
// 对象存储相关错误 (1090-1099)
|
||||
CodeStorageNotConfigured = 1090 // 对象存储服务未配置
|
||||
CodeStorageUploadFailed = 1091 // 文件上传失败
|
||||
CodeStorageDownloadFailed = 1092 // 文件下载失败
|
||||
CodeStorageFileNotFound = 1093 // 文件不存在
|
||||
CodeStorageInvalidPurpose = 1094 // 不支持的文件用途
|
||||
CodeStorageInvalidFileType = 1095 // 不支持的文件类型
|
||||
|
||||
// 服务端错误 (2000-2999) -> 5xx HTTP 状态码
|
||||
CodeInternalError = 2001 // 内部服务器错误
|
||||
CodeDatabaseError = 2002 // 数据库错误
|
||||
@@ -130,6 +138,12 @@ var allErrorCodes = []int{
|
||||
CodeNotDirectSubordinate,
|
||||
CodeCannotAllocateToSelf,
|
||||
CodeCannotRecallFromSelf,
|
||||
CodeStorageNotConfigured,
|
||||
CodeStorageUploadFailed,
|
||||
CodeStorageDownloadFailed,
|
||||
CodeStorageFileNotFound,
|
||||
CodeStorageInvalidPurpose,
|
||||
CodeStorageInvalidFileType,
|
||||
CodeInternalError,
|
||||
CodeDatabaseError,
|
||||
CodeRedisError,
|
||||
@@ -194,6 +208,12 @@ var errorMessages = map[int]string{
|
||||
CodeNotDirectSubordinate: "只能操作直属下级店铺",
|
||||
CodeCannotAllocateToSelf: "不能分配给自己",
|
||||
CodeCannotRecallFromSelf: "不能从自己回收",
|
||||
CodeStorageNotConfigured: "对象存储服务未配置",
|
||||
CodeStorageUploadFailed: "文件上传失败",
|
||||
CodeStorageDownloadFailed: "文件下载失败",
|
||||
CodeStorageFileNotFound: "文件不存在",
|
||||
CodeStorageInvalidPurpose: "不支持的文件用途",
|
||||
CodeStorageInvalidFileType: "不支持的文件类型",
|
||||
CodeInvalidCredentials: "用户名或密码错误",
|
||||
CodeAccountLocked: "账号已锁定",
|
||||
CodePasswordExpired: "密码已过期",
|
||||
|
||||
@@ -4,6 +4,9 @@ import (
|
||||
"encoding/json"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"reflect"
|
||||
"strconv"
|
||||
"strings"
|
||||
|
||||
"github.com/swaggest/openapi-go/openapi3"
|
||||
"gopkg.in/yaml.v3"
|
||||
@@ -85,26 +88,37 @@ func (g *Generator) addErrorResponseSchema() {
|
||||
g.Reflector.Spec.ComponentsEns().SchemasEns().WithMapOfSchemaOrRefValuesItem("ErrorResponse", errorSchema)
|
||||
}
|
||||
|
||||
// ptrString 返回字符串指针
|
||||
func ptrString(s string) *string {
|
||||
return &s
|
||||
}
|
||||
|
||||
// FileUploadField 定义文件上传字段
|
||||
type FileUploadField struct {
|
||||
Name string
|
||||
Description string
|
||||
Required bool
|
||||
}
|
||||
|
||||
// AddOperation 向 OpenAPI 规范中添加一个操作
|
||||
// 参数:
|
||||
// - method: HTTP 方法(GET, POST, PUT, DELETE 等)
|
||||
// - path: API 路径
|
||||
// - summary: 操作摘要
|
||||
// - description: 详细说明,支持 Markdown 语法(可为空)
|
||||
// - input: 请求参数结构体(可为 nil)
|
||||
// - output: 响应结构体(可为 nil)
|
||||
// - tags: 标签列表
|
||||
// - requiresAuth: 是否需要认证
|
||||
func (g *Generator) AddOperation(method, path, summary string, input interface{}, output interface{}, requiresAuth bool, tags ...string) {
|
||||
func (g *Generator) AddOperation(method, path, summary, description string, input interface{}, output interface{}, requiresAuth bool, tags ...string) {
|
||||
op := openapi3.Operation{
|
||||
Summary: &summary,
|
||||
Tags: tags,
|
||||
}
|
||||
|
||||
if description != "" {
|
||||
op.Description = &description
|
||||
}
|
||||
|
||||
// 反射输入 (请求参数/Body)
|
||||
if input != nil {
|
||||
// SetRequest 根据结构体标签自动检测 Body、Query 或 Path 参数
|
||||
@@ -134,6 +148,166 @@ func (g *Generator) AddOperation(method, path, summary string, input interface{}
|
||||
}
|
||||
}
|
||||
|
||||
// AddMultipartOperation 添加支持文件上传的 multipart/form-data 操作
|
||||
func (g *Generator) AddMultipartOperation(method, path, summary, description string, input interface{}, output interface{}, requiresAuth bool, fileFields []FileUploadField, tags ...string) {
|
||||
op := openapi3.Operation{
|
||||
Summary: &summary,
|
||||
Tags: tags,
|
||||
}
|
||||
|
||||
if description != "" {
|
||||
op.Description = &description
|
||||
}
|
||||
|
||||
objectType := openapi3.SchemaType("object")
|
||||
stringType := openapi3.SchemaType("string")
|
||||
integerType := openapi3.SchemaType("integer")
|
||||
binaryFormat := "binary"
|
||||
|
||||
properties := make(map[string]openapi3.SchemaOrRef)
|
||||
var requiredFields []string
|
||||
|
||||
for _, f := range fileFields {
|
||||
properties[f.Name] = openapi3.SchemaOrRef{
|
||||
Schema: &openapi3.Schema{
|
||||
Type: &stringType,
|
||||
Format: &binaryFormat,
|
||||
Description: ptrString(f.Description),
|
||||
},
|
||||
}
|
||||
if f.Required {
|
||||
requiredFields = append(requiredFields, f.Name)
|
||||
}
|
||||
}
|
||||
|
||||
if input != nil {
|
||||
formFields := parseFormFields(input)
|
||||
for _, field := range formFields {
|
||||
var schemaType *openapi3.SchemaType
|
||||
switch field.Type {
|
||||
case "integer":
|
||||
schemaType = &integerType
|
||||
default:
|
||||
schemaType = &stringType
|
||||
}
|
||||
schema := &openapi3.Schema{
|
||||
Type: schemaType,
|
||||
Description: ptrString(field.Description),
|
||||
}
|
||||
if field.Min != nil {
|
||||
schema.Minimum = field.Min
|
||||
}
|
||||
if field.MaxLength != nil {
|
||||
schema.MaxLength = field.MaxLength
|
||||
}
|
||||
properties[field.Name] = openapi3.SchemaOrRef{Schema: schema}
|
||||
if field.Required {
|
||||
requiredFields = append(requiredFields, field.Name)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
op.RequestBody = &openapi3.RequestBodyOrRef{
|
||||
RequestBody: &openapi3.RequestBody{
|
||||
Required: ptrBool(true),
|
||||
Content: map[string]openapi3.MediaType{
|
||||
"multipart/form-data": {
|
||||
Schema: &openapi3.SchemaOrRef{
|
||||
Schema: &openapi3.Schema{
|
||||
Type: &objectType,
|
||||
Properties: properties,
|
||||
Required: requiredFields,
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
if output != nil {
|
||||
if err := g.Reflector.SetJSONResponse(&op, output, 200); err != nil {
|
||||
panic(err)
|
||||
}
|
||||
}
|
||||
|
||||
if requiresAuth {
|
||||
g.addSecurityRequirement(&op)
|
||||
}
|
||||
|
||||
g.addStandardErrorResponses(&op, requiresAuth)
|
||||
|
||||
if err := g.Reflector.Spec.AddOperation(method, path, op); err != nil {
|
||||
panic(err)
|
||||
}
|
||||
}
|
||||
|
||||
func ptrBool(b bool) *bool {
|
||||
return &b
|
||||
}
|
||||
|
||||
type formFieldInfo struct {
|
||||
Name string
|
||||
Type string
|
||||
Description string
|
||||
Required bool
|
||||
Min *float64
|
||||
MaxLength *int64
|
||||
}
|
||||
|
||||
func parseFormFields(input interface{}) []formFieldInfo {
|
||||
var fields []formFieldInfo
|
||||
t := reflect.TypeOf(input)
|
||||
if t.Kind() == reflect.Ptr {
|
||||
t = t.Elem()
|
||||
}
|
||||
if t.Kind() != reflect.Struct {
|
||||
return fields
|
||||
}
|
||||
|
||||
for i := 0; i < t.NumField(); i++ {
|
||||
field := t.Field(i)
|
||||
|
||||
formTag := field.Tag.Get("form")
|
||||
if formTag == "" || formTag == "-" {
|
||||
continue
|
||||
}
|
||||
|
||||
info := formFieldInfo{
|
||||
Name: formTag,
|
||||
Description: field.Tag.Get("description"),
|
||||
}
|
||||
|
||||
switch field.Type.Kind() {
|
||||
case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64,
|
||||
reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64:
|
||||
info.Type = "integer"
|
||||
default:
|
||||
info.Type = "string"
|
||||
}
|
||||
|
||||
validateTag := field.Tag.Get("validate")
|
||||
if strings.Contains(validateTag, "required") {
|
||||
info.Required = true
|
||||
}
|
||||
|
||||
if minStr := field.Tag.Get("minimum"); minStr != "" {
|
||||
if min, err := strconv.ParseFloat(minStr, 64); err == nil {
|
||||
info.Min = &min
|
||||
}
|
||||
}
|
||||
|
||||
if maxLenStr := field.Tag.Get("maxLength"); maxLenStr != "" {
|
||||
if maxLen, err := strconv.ParseInt(maxLenStr, 10, 64); err == nil {
|
||||
info.MaxLength = &maxLen
|
||||
}
|
||||
}
|
||||
|
||||
fields = append(fields, info)
|
||||
}
|
||||
|
||||
return fields
|
||||
}
|
||||
|
||||
// addSecurityRequirement 为操作添加认证要求
|
||||
func (g *Generator) addSecurityRequirement(op *openapi3.Operation) {
|
||||
op.Security = []map[string][]string{
|
||||
|
||||
@@ -9,23 +9,24 @@ import (
|
||||
"github.com/break/junhong_cmp_fiber/internal/store/postgres"
|
||||
"github.com/break/junhong_cmp_fiber/internal/task"
|
||||
"github.com/break/junhong_cmp_fiber/pkg/constants"
|
||||
"github.com/break/junhong_cmp_fiber/pkg/storage"
|
||||
)
|
||||
|
||||
// Handler 任务处理器注册
|
||||
type Handler struct {
|
||||
mux *asynq.ServeMux
|
||||
logger *zap.Logger
|
||||
db *gorm.DB
|
||||
redis *redis.Client
|
||||
mux *asynq.ServeMux
|
||||
logger *zap.Logger
|
||||
db *gorm.DB
|
||||
redis *redis.Client
|
||||
storage *storage.Service
|
||||
}
|
||||
|
||||
// NewHandler 创建任务处理器
|
||||
func NewHandler(db *gorm.DB, redis *redis.Client, logger *zap.Logger) *Handler {
|
||||
func NewHandler(db *gorm.DB, redis *redis.Client, storageSvc *storage.Service, logger *zap.Logger) *Handler {
|
||||
return &Handler{
|
||||
mux: asynq.NewServeMux(),
|
||||
logger: logger,
|
||||
db: db,
|
||||
redis: redis,
|
||||
mux: asynq.NewServeMux(),
|
||||
logger: logger,
|
||||
db: db,
|
||||
redis: redis,
|
||||
storage: storageSvc,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -53,7 +54,7 @@ func (h *Handler) RegisterHandlers() *asynq.ServeMux {
|
||||
func (h *Handler) registerIotCardImportHandler() {
|
||||
importTaskStore := postgres.NewIotCardImportTaskStore(h.db, h.redis)
|
||||
iotCardStore := postgres.NewIotCardStore(h.db, h.redis)
|
||||
iotCardImportHandler := task.NewIotCardImportHandler(h.db, h.redis, importTaskStore, iotCardStore, h.logger)
|
||||
iotCardImportHandler := task.NewIotCardImportHandler(h.db, h.redis, importTaskStore, iotCardStore, h.storage, h.logger)
|
||||
|
||||
h.mux.HandleFunc(constants.TaskTypeIotCardImport, iotCardImportHandler.HandleIotCardImport)
|
||||
h.logger.Info("注册 IoT 卡导入任务处理器", zap.String("task_type", constants.TaskTypeIotCardImport))
|
||||
|
||||
184
pkg/storage/s3.go
Normal file
184
pkg/storage/s3.go
Normal file
@@ -0,0 +1,184 @@
|
||||
package storage
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"io"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/aws/aws-sdk-go/aws"
|
||||
"github.com/aws/aws-sdk-go/aws/credentials"
|
||||
"github.com/aws/aws-sdk-go/aws/session"
|
||||
"github.com/aws/aws-sdk-go/service/s3"
|
||||
"github.com/aws/aws-sdk-go/service/s3/s3manager"
|
||||
|
||||
"github.com/break/junhong_cmp_fiber/pkg/config"
|
||||
)
|
||||
|
||||
type S3Provider struct {
|
||||
client *s3.S3
|
||||
uploader *s3manager.Uploader
|
||||
bucket string
|
||||
tempDir string
|
||||
}
|
||||
|
||||
func NewS3Provider(cfg *config.StorageConfig) (*S3Provider, error) {
|
||||
if cfg.S3.Endpoint == "" || cfg.S3.Bucket == "" {
|
||||
return nil, fmt.Errorf("S3 配置不完整:endpoint 和 bucket 必填")
|
||||
}
|
||||
|
||||
if cfg.S3.AccessKeyID == "" || cfg.S3.SecretAccessKey == "" {
|
||||
return nil, fmt.Errorf("S3 凭证未配置:access_key_id 和 secret_access_key 必填")
|
||||
}
|
||||
|
||||
sess, err := session.NewSession(&aws.Config{
|
||||
Endpoint: aws.String(cfg.S3.Endpoint),
|
||||
Region: aws.String(cfg.S3.Region),
|
||||
Credentials: credentials.NewStaticCredentials(cfg.S3.AccessKeyID, cfg.S3.SecretAccessKey, ""),
|
||||
DisableSSL: aws.Bool(!cfg.S3.UseSSL),
|
||||
S3ForcePathStyle: aws.Bool(cfg.S3.PathStyle),
|
||||
})
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("创建 S3 session 失败: %w", err)
|
||||
}
|
||||
|
||||
tempDir := cfg.TempDir
|
||||
if tempDir == "" {
|
||||
tempDir = "/tmp/junhong-storage"
|
||||
}
|
||||
|
||||
return &S3Provider{
|
||||
client: s3.New(sess),
|
||||
uploader: s3manager.NewUploader(sess),
|
||||
bucket: cfg.S3.Bucket,
|
||||
tempDir: tempDir,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (p *S3Provider) Upload(ctx context.Context, key string, reader io.Reader, contentType string) error {
|
||||
input := &s3manager.UploadInput{
|
||||
Bucket: aws.String(p.bucket),
|
||||
Key: aws.String(key),
|
||||
Body: reader,
|
||||
ContentType: aws.String(contentType),
|
||||
}
|
||||
|
||||
_, err := p.uploader.UploadWithContext(ctx, input)
|
||||
if err != nil {
|
||||
return fmt.Errorf("上传文件失败: %w", err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (p *S3Provider) Download(ctx context.Context, key string, writer io.Writer) error {
|
||||
input := &s3.GetObjectInput{
|
||||
Bucket: aws.String(p.bucket),
|
||||
Key: aws.String(key),
|
||||
}
|
||||
|
||||
result, err := p.client.GetObjectWithContext(ctx, input)
|
||||
if err != nil {
|
||||
if strings.Contains(err.Error(), "NoSuchKey") {
|
||||
return fmt.Errorf("文件不存在: %s", key)
|
||||
}
|
||||
return fmt.Errorf("下载文件失败: %w", err)
|
||||
}
|
||||
defer result.Body.Close()
|
||||
|
||||
_, err = io.Copy(writer, result.Body)
|
||||
if err != nil {
|
||||
return fmt.Errorf("写入文件内容失败: %w", err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (p *S3Provider) DownloadToTemp(ctx context.Context, key string) (string, func(), error) {
|
||||
ext := filepath.Ext(key)
|
||||
if ext == "" {
|
||||
ext = ".tmp"
|
||||
}
|
||||
|
||||
tempFile, err := os.CreateTemp(p.tempDir, "download-*"+ext)
|
||||
if err != nil {
|
||||
return "", nil, fmt.Errorf("创建临时文件失败: %w", err)
|
||||
}
|
||||
tempPath := tempFile.Name()
|
||||
|
||||
cleanup := func() {
|
||||
_ = os.Remove(tempPath)
|
||||
}
|
||||
|
||||
if err := p.Download(ctx, key, tempFile); err != nil {
|
||||
tempFile.Close()
|
||||
cleanup()
|
||||
return "", nil, err
|
||||
}
|
||||
|
||||
if err := tempFile.Close(); err != nil {
|
||||
cleanup()
|
||||
return "", nil, fmt.Errorf("关闭临时文件失败: %w", err)
|
||||
}
|
||||
|
||||
return tempPath, cleanup, nil
|
||||
}
|
||||
|
||||
func (p *S3Provider) Delete(ctx context.Context, key string) error {
|
||||
input := &s3.DeleteObjectInput{
|
||||
Bucket: aws.String(p.bucket),
|
||||
Key: aws.String(key),
|
||||
}
|
||||
|
||||
_, err := p.client.DeleteObjectWithContext(ctx, input)
|
||||
if err != nil {
|
||||
return fmt.Errorf("删除文件失败: %w", err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (p *S3Provider) Exists(ctx context.Context, key string) (bool, error) {
|
||||
input := &s3.HeadObjectInput{
|
||||
Bucket: aws.String(p.bucket),
|
||||
Key: aws.String(key),
|
||||
}
|
||||
|
||||
_, err := p.client.HeadObjectWithContext(ctx, input)
|
||||
if err != nil {
|
||||
if strings.Contains(err.Error(), "NotFound") || strings.Contains(err.Error(), "404") {
|
||||
return false, nil
|
||||
}
|
||||
return false, fmt.Errorf("检查文件存在性失败: %w", err)
|
||||
}
|
||||
return true, nil
|
||||
}
|
||||
|
||||
func (p *S3Provider) GetUploadURL(ctx context.Context, key string, contentType string, expires time.Duration) (string, error) {
|
||||
input := &s3.PutObjectInput{
|
||||
Bucket: aws.String(p.bucket),
|
||||
Key: aws.String(key),
|
||||
ContentType: aws.String(contentType),
|
||||
}
|
||||
|
||||
req, _ := p.client.PutObjectRequest(input)
|
||||
url, err := req.Presign(expires)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("生成上传预签名 URL 失败: %w", err)
|
||||
}
|
||||
return url, nil
|
||||
}
|
||||
|
||||
func (p *S3Provider) GetDownloadURL(ctx context.Context, key string, expires time.Duration) (string, error) {
|
||||
input := &s3.GetObjectInput{
|
||||
Bucket: aws.String(p.bucket),
|
||||
Key: aws.String(key),
|
||||
}
|
||||
|
||||
req, _ := p.client.GetObjectRequest(input)
|
||||
url, err := req.Presign(expires)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("生成下载预签名 URL 失败: %w", err)
|
||||
}
|
||||
return url, nil
|
||||
}
|
||||
112
pkg/storage/service.go
Normal file
112
pkg/storage/service.go
Normal file
@@ -0,0 +1,112 @@
|
||||
package storage
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/google/uuid"
|
||||
|
||||
"github.com/break/junhong_cmp_fiber/pkg/config"
|
||||
)
|
||||
|
||||
type Service struct {
|
||||
provider Provider
|
||||
config *config.StorageConfig
|
||||
}
|
||||
|
||||
func NewService(provider Provider, cfg *config.StorageConfig) *Service {
|
||||
return &Service{
|
||||
provider: provider,
|
||||
config: cfg,
|
||||
}
|
||||
}
|
||||
|
||||
func (s *Service) GenerateFileKey(purpose, fileName string) (string, error) {
|
||||
mapping, ok := PurposeMappings[purpose]
|
||||
if !ok {
|
||||
return "", fmt.Errorf("不支持的文件用途: %s", purpose)
|
||||
}
|
||||
|
||||
ext := filepath.Ext(fileName)
|
||||
if ext == "" {
|
||||
ext = ".bin"
|
||||
}
|
||||
|
||||
now := time.Now()
|
||||
id := uuid.New().String()
|
||||
|
||||
key := fmt.Sprintf("%s/%04d/%02d/%02d/%s%s",
|
||||
mapping.Prefix,
|
||||
now.Year(),
|
||||
now.Month(),
|
||||
now.Day(),
|
||||
id,
|
||||
strings.ToLower(ext),
|
||||
)
|
||||
|
||||
return key, nil
|
||||
}
|
||||
|
||||
func (s *Service) GetUploadURL(ctx context.Context, purpose, fileName, contentType string) (*PresignResult, error) {
|
||||
fileKey, err := s.GenerateFileKey(purpose, fileName)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if contentType == "" {
|
||||
if mapping, ok := PurposeMappings[purpose]; ok && mapping.ContentType != "" {
|
||||
contentType = mapping.ContentType
|
||||
} else {
|
||||
contentType = "application/octet-stream"
|
||||
}
|
||||
}
|
||||
|
||||
expires := s.config.Presign.UploadExpires
|
||||
if expires == 0 {
|
||||
expires = 15 * time.Minute
|
||||
}
|
||||
|
||||
url, err := s.provider.GetUploadURL(ctx, fileKey, contentType, expires)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return &PresignResult{
|
||||
URL: url,
|
||||
FileKey: fileKey,
|
||||
ExpiresIn: int(expires.Seconds()),
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (s *Service) GetDownloadURL(ctx context.Context, fileKey string) (*PresignResult, error) {
|
||||
expires := s.config.Presign.DownloadExpires
|
||||
if expires == 0 {
|
||||
expires = 24 * time.Hour
|
||||
}
|
||||
|
||||
url, err := s.provider.GetDownloadURL(ctx, fileKey, expires)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return &PresignResult{
|
||||
URL: url,
|
||||
FileKey: fileKey,
|
||||
ExpiresIn: int(expires.Seconds()),
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (s *Service) DownloadToTemp(ctx context.Context, fileKey string) (string, func(), error) {
|
||||
return s.provider.DownloadToTemp(ctx, fileKey)
|
||||
}
|
||||
|
||||
func (s *Service) Provider() Provider {
|
||||
return s.provider
|
||||
}
|
||||
|
||||
func (s *Service) Bucket() string {
|
||||
return s.config.S3.Bucket
|
||||
}
|
||||
17
pkg/storage/storage.go
Normal file
17
pkg/storage/storage.go
Normal file
@@ -0,0 +1,17 @@
|
||||
package storage
|
||||
|
||||
import (
|
||||
"context"
|
||||
"io"
|
||||
"time"
|
||||
)
|
||||
|
||||
type Provider interface {
|
||||
Upload(ctx context.Context, key string, reader io.Reader, contentType string) error
|
||||
Download(ctx context.Context, key string, writer io.Writer) error
|
||||
DownloadToTemp(ctx context.Context, key string) (localPath string, cleanup func(), err error)
|
||||
Delete(ctx context.Context, key string) error
|
||||
Exists(ctx context.Context, key string) (bool, error)
|
||||
GetUploadURL(ctx context.Context, key string, contentType string, expires time.Duration) (string, error)
|
||||
GetDownloadURL(ctx context.Context, key string, expires time.Duration) (string, error)
|
||||
}
|
||||
18
pkg/storage/types.go
Normal file
18
pkg/storage/types.go
Normal file
@@ -0,0 +1,18 @@
|
||||
package storage
|
||||
|
||||
type PresignResult struct {
|
||||
URL string `json:"url"`
|
||||
FileKey string `json:"file_key"`
|
||||
ExpiresIn int `json:"expires_in"`
|
||||
}
|
||||
|
||||
type PurposeMapping struct {
|
||||
Prefix string
|
||||
ContentType string
|
||||
}
|
||||
|
||||
var PurposeMappings = map[string]PurposeMapping{
|
||||
"iot_import": {Prefix: "imports", ContentType: "text/csv"},
|
||||
"export": {Prefix: "exports", ContentType: "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet"},
|
||||
"attachment": {Prefix: "attachments", ContentType: ""},
|
||||
}
|
||||
@@ -2,33 +2,49 @@ package utils
|
||||
|
||||
import (
|
||||
"encoding/csv"
|
||||
"errors"
|
||||
"io"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// CardInfo 卡信息(ICCID + MSISDN)
|
||||
type CardInfo struct {
|
||||
ICCID string
|
||||
MSISDN string
|
||||
}
|
||||
|
||||
// CSVParseResult CSV 解析结果
|
||||
type CSVParseResult struct {
|
||||
ICCIDs []string
|
||||
Cards []CardInfo
|
||||
TotalCount int
|
||||
ParseErrors []CSVParseError
|
||||
}
|
||||
|
||||
// CSVParseError CSV 解析错误
|
||||
type CSVParseError struct {
|
||||
Line int
|
||||
ICCID string
|
||||
MSISDN string
|
||||
Reason string
|
||||
}
|
||||
|
||||
func ParseICCIDFromCSV(reader io.Reader) (*CSVParseResult, error) {
|
||||
// ErrInvalidCSVFormat CSV 格式错误
|
||||
var ErrInvalidCSVFormat = errors.New("CSV 文件格式错误:缺少 MSISDN 列,文件必须包含 ICCID 和 MSISDN 两列")
|
||||
|
||||
// ParseCardCSV 解析包含 ICCID 和 MSISDN 两列的 CSV 文件
|
||||
func ParseCardCSV(reader io.Reader) (*CSVParseResult, error) {
|
||||
csvReader := csv.NewReader(reader)
|
||||
csvReader.FieldsPerRecord = -1
|
||||
csvReader.TrimLeadingSpace = true
|
||||
|
||||
result := &CSVParseResult{
|
||||
ICCIDs: make([]string, 0),
|
||||
Cards: make([]CardInfo, 0),
|
||||
ParseErrors: make([]CSVParseError, 0),
|
||||
}
|
||||
|
||||
lineNum := 0
|
||||
headerSkipped := false
|
||||
|
||||
for {
|
||||
record, err := csvReader.Read()
|
||||
if err == io.EOF {
|
||||
@@ -48,23 +64,69 @@ func ParseICCIDFromCSV(reader io.Reader) (*CSVParseResult, error) {
|
||||
continue
|
||||
}
|
||||
|
||||
iccid := strings.TrimSpace(record[0])
|
||||
if iccid == "" {
|
||||
if len(record) < 2 {
|
||||
if lineNum == 1 && !headerSkipped {
|
||||
firstCol := strings.TrimSpace(record[0])
|
||||
if isICCIDHeader(firstCol) {
|
||||
return nil, ErrInvalidCSVFormat
|
||||
}
|
||||
}
|
||||
result.ParseErrors = append(result.ParseErrors, CSVParseError{
|
||||
Line: lineNum,
|
||||
ICCID: strings.TrimSpace(record[0]),
|
||||
Reason: "列数不足:缺少 MSISDN 列",
|
||||
})
|
||||
result.TotalCount++
|
||||
continue
|
||||
}
|
||||
|
||||
if lineNum == 1 && isHeader(iccid) {
|
||||
iccid := strings.TrimSpace(record[0])
|
||||
msisdn := strings.TrimSpace(record[1])
|
||||
|
||||
if lineNum == 1 && !headerSkipped && isHeader(iccid, msisdn) {
|
||||
headerSkipped = true
|
||||
continue
|
||||
}
|
||||
|
||||
result.TotalCount++
|
||||
result.ICCIDs = append(result.ICCIDs, iccid)
|
||||
|
||||
if iccid == "" {
|
||||
result.ParseErrors = append(result.ParseErrors, CSVParseError{
|
||||
Line: lineNum,
|
||||
MSISDN: msisdn,
|
||||
Reason: "ICCID 不能为空",
|
||||
})
|
||||
continue
|
||||
}
|
||||
|
||||
if msisdn == "" {
|
||||
result.ParseErrors = append(result.ParseErrors, CSVParseError{
|
||||
Line: lineNum,
|
||||
ICCID: iccid,
|
||||
Reason: "MSISDN 不能为空",
|
||||
})
|
||||
continue
|
||||
}
|
||||
|
||||
result.Cards = append(result.Cards, CardInfo{
|
||||
ICCID: iccid,
|
||||
MSISDN: msisdn,
|
||||
})
|
||||
}
|
||||
|
||||
return result, nil
|
||||
}
|
||||
|
||||
func isHeader(value string) bool {
|
||||
func isICCIDHeader(value string) bool {
|
||||
lower := strings.ToLower(value)
|
||||
return lower == "iccid" || lower == "卡号" || lower == "号码"
|
||||
}
|
||||
|
||||
func isMSISDNHeader(value string) bool {
|
||||
lower := strings.ToLower(value)
|
||||
return lower == "msisdn" || lower == "接入号" || lower == "手机号" || lower == "电话" || lower == "号码"
|
||||
}
|
||||
|
||||
func isHeader(col1, col2 string) bool {
|
||||
return isICCIDHeader(col1) && isMSISDNHeader(col2)
|
||||
}
|
||||
|
||||
@@ -8,89 +8,133 @@ import (
|
||||
"github.com/stretchr/testify/require"
|
||||
)
|
||||
|
||||
func TestParseICCIDFromCSV(t *testing.T) {
|
||||
func TestParseCardCSV(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
csvContent string
|
||||
wantICCIDs []string
|
||||
wantCards []CardInfo
|
||||
wantTotalCount int
|
||||
wantErrorCount int
|
||||
wantError error
|
||||
}{
|
||||
{
|
||||
name: "单列ICCID无表头",
|
||||
csvContent: "89860012345678901234\n89860012345678901235\n89860012345678901236",
|
||||
wantICCIDs: []string{"89860012345678901234", "89860012345678901235", "89860012345678901236"},
|
||||
wantTotalCount: 3,
|
||||
wantErrorCount: 0,
|
||||
},
|
||||
{
|
||||
name: "单列ICCID有表头-iccid",
|
||||
csvContent: "iccid\n89860012345678901234\n89860012345678901235",
|
||||
wantICCIDs: []string{"89860012345678901234", "89860012345678901235"},
|
||||
name: "标准双列无表头",
|
||||
csvContent: "89860012345678901234,13800000001\n89860012345678901235,13800000002",
|
||||
wantCards: []CardInfo{
|
||||
{ICCID: "89860012345678901234", MSISDN: "13800000001"},
|
||||
{ICCID: "89860012345678901235", MSISDN: "13800000002"},
|
||||
},
|
||||
wantTotalCount: 2,
|
||||
wantErrorCount: 0,
|
||||
},
|
||||
{
|
||||
name: "单列ICCID有表头-ICCID大写",
|
||||
csvContent: "ICCID\n89860012345678901234",
|
||||
wantICCIDs: []string{"89860012345678901234"},
|
||||
name: "标准双列有表头-英文",
|
||||
csvContent: "iccid,msisdn\n89860012345678901234,13800000001\n89860012345678901235,13800000002",
|
||||
wantCards: []CardInfo{
|
||||
{ICCID: "89860012345678901234", MSISDN: "13800000001"},
|
||||
{ICCID: "89860012345678901235", MSISDN: "13800000002"},
|
||||
},
|
||||
wantTotalCount: 2,
|
||||
wantErrorCount: 0,
|
||||
},
|
||||
{
|
||||
name: "标准双列有表头-中文",
|
||||
csvContent: "卡号,接入号\n89860012345678901234,13800000001",
|
||||
wantCards: []CardInfo{
|
||||
{ICCID: "89860012345678901234", MSISDN: "13800000001"},
|
||||
},
|
||||
wantTotalCount: 1,
|
||||
wantErrorCount: 0,
|
||||
},
|
||||
{
|
||||
name: "单列ICCID有表头-卡号",
|
||||
csvContent: "卡号\n89860012345678901234",
|
||||
wantICCIDs: []string{"89860012345678901234"},
|
||||
name: "标准双列有表头-手机号",
|
||||
csvContent: "ICCID,手机号\n89860012345678901234,13800000001",
|
||||
wantCards: []CardInfo{
|
||||
{ICCID: "89860012345678901234", MSISDN: "13800000001"},
|
||||
},
|
||||
wantTotalCount: 1,
|
||||
wantErrorCount: 0,
|
||||
},
|
||||
{
|
||||
name: "单列ICCID有表头-号码",
|
||||
csvContent: "号码\n89860012345678901234",
|
||||
wantICCIDs: []string{"89860012345678901234"},
|
||||
wantTotalCount: 1,
|
||||
name: "单列CSV格式拒绝-有表头",
|
||||
csvContent: "iccid\n89860012345678901234",
|
||||
wantCards: nil,
|
||||
wantTotalCount: 0,
|
||||
wantErrorCount: 0,
|
||||
wantError: ErrInvalidCSVFormat,
|
||||
},
|
||||
{
|
||||
name: "单列CSV格式-无表头记录错误",
|
||||
csvContent: "89860012345678901234\n89860012345678901235",
|
||||
wantCards: []CardInfo{},
|
||||
wantTotalCount: 2,
|
||||
wantErrorCount: 2,
|
||||
},
|
||||
{
|
||||
name: "MSISDN为空记录失败",
|
||||
csvContent: "iccid,msisdn\n89860012345678901234,13800000001\n89860012345678901235,",
|
||||
wantCards: []CardInfo{{ICCID: "89860012345678901234", MSISDN: "13800000001"}},
|
||||
wantTotalCount: 2,
|
||||
wantErrorCount: 1,
|
||||
},
|
||||
{
|
||||
name: "ICCID为空记录失败",
|
||||
csvContent: "iccid,msisdn\n89860012345678901234,13800000001\n,13800000002",
|
||||
wantCards: []CardInfo{{ICCID: "89860012345678901234", MSISDN: "13800000001"}},
|
||||
wantTotalCount: 2,
|
||||
wantErrorCount: 1,
|
||||
},
|
||||
{
|
||||
name: "空文件",
|
||||
csvContent: "",
|
||||
wantICCIDs: []string{},
|
||||
wantCards: []CardInfo{},
|
||||
wantTotalCount: 0,
|
||||
wantErrorCount: 0,
|
||||
},
|
||||
{
|
||||
name: "只有表头",
|
||||
csvContent: "iccid",
|
||||
wantICCIDs: []string{},
|
||||
csvContent: "iccid,msisdn",
|
||||
wantCards: []CardInfo{},
|
||||
wantTotalCount: 0,
|
||||
wantErrorCount: 0,
|
||||
},
|
||||
{
|
||||
name: "包含空行",
|
||||
csvContent: "89860012345678901234\n\n89860012345678901235\n \n89860012345678901236",
|
||||
wantICCIDs: []string{"89860012345678901234", "89860012345678901235", "89860012345678901236"},
|
||||
wantTotalCount: 3,
|
||||
wantErrorCount: 0,
|
||||
},
|
||||
{
|
||||
name: "ICCID前后有空格",
|
||||
csvContent: " 89860012345678901234 \n89860012345678901235",
|
||||
wantICCIDs: []string{"89860012345678901234", "89860012345678901235"},
|
||||
name: "包含空行",
|
||||
csvContent: "89860012345678901234,13800000001\n\n89860012345678901235,13800000002",
|
||||
wantCards: []CardInfo{
|
||||
{ICCID: "89860012345678901234", MSISDN: "13800000001"},
|
||||
{ICCID: "89860012345678901235", MSISDN: "13800000002"},
|
||||
},
|
||||
wantTotalCount: 2,
|
||||
wantErrorCount: 0,
|
||||
},
|
||||
{
|
||||
name: "多列CSV只取第一列",
|
||||
csvContent: "89860012345678901234,额外数据,更多数据\n89860012345678901235,忽略,忽略",
|
||||
wantICCIDs: []string{"89860012345678901234", "89860012345678901235"},
|
||||
name: "ICCID和MSISDN前后有空格",
|
||||
csvContent: " 89860012345678901234 , 13800000001 ",
|
||||
wantCards: []CardInfo{
|
||||
{ICCID: "89860012345678901234", MSISDN: "13800000001"},
|
||||
},
|
||||
wantTotalCount: 1,
|
||||
wantErrorCount: 0,
|
||||
},
|
||||
{
|
||||
name: "多于两列只取前两列",
|
||||
csvContent: "89860012345678901234,13800000001,额外数据\n89860012345678901235,13800000002,忽略",
|
||||
wantCards: []CardInfo{
|
||||
{ICCID: "89860012345678901234", MSISDN: "13800000001"},
|
||||
{ICCID: "89860012345678901235", MSISDN: "13800000002"},
|
||||
},
|
||||
wantTotalCount: 2,
|
||||
wantErrorCount: 0,
|
||||
},
|
||||
{
|
||||
name: "Windows换行符CRLF",
|
||||
csvContent: "89860012345678901234\r\n89860012345678901235\r\n89860012345678901236",
|
||||
wantICCIDs: []string{"89860012345678901234", "89860012345678901235", "89860012345678901236"},
|
||||
wantTotalCount: 3,
|
||||
name: "Windows换行符CRLF",
|
||||
csvContent: "89860012345678901234,13800000001\r\n89860012345678901235,13800000002",
|
||||
wantCards: []CardInfo{
|
||||
{ICCID: "89860012345678901234", MSISDN: "13800000001"},
|
||||
{ICCID: "89860012345678901235", MSISDN: "13800000002"},
|
||||
},
|
||||
wantTotalCount: 2,
|
||||
wantErrorCount: 0,
|
||||
},
|
||||
}
|
||||
@@ -98,34 +142,78 @@ func TestParseICCIDFromCSV(t *testing.T) {
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
reader := strings.NewReader(tt.csvContent)
|
||||
result, err := ParseICCIDFromCSV(reader)
|
||||
result, err := ParseCardCSV(reader)
|
||||
|
||||
if tt.wantError != nil {
|
||||
require.ErrorIs(t, err, tt.wantError)
|
||||
return
|
||||
}
|
||||
|
||||
require.NoError(t, err)
|
||||
assert.Equal(t, tt.wantICCIDs, result.ICCIDs, "ICCIDs 不匹配")
|
||||
assert.Equal(t, tt.wantCards, result.Cards, "Cards 不匹配")
|
||||
assert.Equal(t, tt.wantTotalCount, result.TotalCount, "TotalCount 不匹配")
|
||||
assert.Equal(t, tt.wantErrorCount, len(result.ParseErrors), "ParseErrors 数量不匹配")
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestParseCardCSV_ErrorDetails(t *testing.T) {
|
||||
t.Run("MSISDN为空时记录详细错误", func(t *testing.T) {
|
||||
csvContent := "iccid,msisdn\n89860012345678901234,"
|
||||
reader := strings.NewReader(csvContent)
|
||||
result, err := ParseCardCSV(reader)
|
||||
|
||||
require.NoError(t, err)
|
||||
require.Len(t, result.ParseErrors, 1)
|
||||
assert.Equal(t, 2, result.ParseErrors[0].Line)
|
||||
assert.Equal(t, "89860012345678901234", result.ParseErrors[0].ICCID)
|
||||
assert.Equal(t, "MSISDN 不能为空", result.ParseErrors[0].Reason)
|
||||
})
|
||||
|
||||
t.Run("ICCID为空时记录详细错误", func(t *testing.T) {
|
||||
csvContent := "iccid,msisdn\n,13800000001"
|
||||
reader := strings.NewReader(csvContent)
|
||||
result, err := ParseCardCSV(reader)
|
||||
|
||||
require.NoError(t, err)
|
||||
require.Len(t, result.ParseErrors, 1)
|
||||
assert.Equal(t, 2, result.ParseErrors[0].Line)
|
||||
assert.Equal(t, "13800000001", result.ParseErrors[0].MSISDN)
|
||||
assert.Equal(t, "ICCID 不能为空", result.ParseErrors[0].Reason)
|
||||
})
|
||||
|
||||
t.Run("列数不足时记录详细错误", func(t *testing.T) {
|
||||
csvContent := "89860012345678901234"
|
||||
reader := strings.NewReader(csvContent)
|
||||
result, err := ParseCardCSV(reader)
|
||||
|
||||
require.NoError(t, err)
|
||||
require.Len(t, result.ParseErrors, 1)
|
||||
assert.Equal(t, 1, result.ParseErrors[0].Line)
|
||||
assert.Equal(t, "89860012345678901234", result.ParseErrors[0].ICCID)
|
||||
assert.Contains(t, result.ParseErrors[0].Reason, "列数不足")
|
||||
})
|
||||
}
|
||||
|
||||
func TestIsHeader(t *testing.T) {
|
||||
tests := []struct {
|
||||
value string
|
||||
col1 string
|
||||
col2 string
|
||||
expected bool
|
||||
}{
|
||||
{"iccid", true},
|
||||
{"ICCID", true},
|
||||
{"Iccid", true},
|
||||
{"卡号", true},
|
||||
{"号码", true},
|
||||
{"89860012345678901234", false},
|
||||
{"", false},
|
||||
{"id", false},
|
||||
{"card", false},
|
||||
{"iccid", "msisdn", true},
|
||||
{"ICCID", "MSISDN", true},
|
||||
{"卡号", "接入号", true},
|
||||
{"号码", "手机号", true},
|
||||
{"iccid", "电话", true},
|
||||
{"89860012345678901234", "13800000001", false},
|
||||
{"iccid", "", false},
|
||||
{"", "msisdn", false},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.value, func(t *testing.T) {
|
||||
result := isHeader(tt.value)
|
||||
t.Run(tt.col1+"_"+tt.col2, func(t *testing.T) {
|
||||
result := isHeader(tt.col1, tt.col2)
|
||||
assert.Equal(t, tt.expected, result)
|
||||
})
|
||||
}
|
||||
|
||||
106
scripts/run-local.sh
Executable file
106
scripts/run-local.sh
Executable file
@@ -0,0 +1,106 @@
|
||||
#!/bin/bash
|
||||
# ============================================================================
|
||||
# 君鸿卡管系统 - 本地开发快速启动脚本
|
||||
# ============================================================================
|
||||
# 使用方法:
|
||||
# ./scripts/run-local.sh # 启动 API 服务
|
||||
# ./scripts/run-local.sh worker # 启动 Worker 服务
|
||||
# ./scripts/run-local.sh both # 同时启动 API 和 Worker
|
||||
# ============================================================================
|
||||
|
||||
set -e
|
||||
|
||||
# 颜色定义
|
||||
RED='\033[0;31m'
|
||||
GREEN='\033[0;32m'
|
||||
YELLOW='\033[1;33m'
|
||||
BLUE='\033[0;34m'
|
||||
NC='\033[0m'
|
||||
|
||||
ENV_FILE=".env.local"
|
||||
|
||||
# 检查环境配置文件
|
||||
check_env() {
|
||||
if [ ! -f "$ENV_FILE" ]; then
|
||||
echo -e "${RED}[ERROR]${NC} 未找到环境配置文件: $ENV_FILE"
|
||||
echo ""
|
||||
echo "请先运行环境配置向导:"
|
||||
echo -e " ${BLUE}./scripts/setup-env.sh${NC}"
|
||||
exit 1
|
||||
fi
|
||||
}
|
||||
|
||||
# 加载环境变量
|
||||
load_env() {
|
||||
echo -e "${BLUE}[INFO]${NC} 加载环境变量: $ENV_FILE"
|
||||
source "$ENV_FILE"
|
||||
}
|
||||
|
||||
# 创建日志目录
|
||||
ensure_logs_dir() {
|
||||
if [ ! -d "logs" ]; then
|
||||
mkdir -p logs
|
||||
echo -e "${GREEN}[OK]${NC} 创建日志目录: logs/"
|
||||
fi
|
||||
}
|
||||
|
||||
# 启动 API 服务
|
||||
start_api() {
|
||||
echo -e "\n${GREEN}========================================${NC}"
|
||||
echo -e "${GREEN} 启动 API 服务${NC}"
|
||||
echo -e "${GREEN}========================================${NC}\n"
|
||||
go run cmd/api/main.go
|
||||
}
|
||||
|
||||
# 启动 Worker 服务
|
||||
start_worker() {
|
||||
echo -e "\n${GREEN}========================================${NC}"
|
||||
echo -e "${GREEN} 启动 Worker 服务${NC}"
|
||||
echo -e "${GREEN}========================================${NC}\n"
|
||||
go run cmd/worker/main.go
|
||||
}
|
||||
|
||||
# 同时启动两个服务
|
||||
start_both() {
|
||||
echo -e "\n${GREEN}========================================${NC}"
|
||||
echo -e "${GREEN} 同时启动 API 和 Worker 服务${NC}"
|
||||
echo -e "${GREEN}========================================${NC}\n"
|
||||
|
||||
# 后台启动 worker
|
||||
go run cmd/worker/main.go &
|
||||
WORKER_PID=$!
|
||||
echo -e "${GREEN}[OK]${NC} Worker 服务已启动 (PID: $WORKER_PID)"
|
||||
|
||||
# 前台启动 api
|
||||
trap "kill $WORKER_PID 2>/dev/null" EXIT
|
||||
go run cmd/api/main.go
|
||||
}
|
||||
|
||||
# 主流程
|
||||
main() {
|
||||
check_env
|
||||
load_env
|
||||
ensure_logs_dir
|
||||
|
||||
case "${1:-api}" in
|
||||
api)
|
||||
start_api
|
||||
;;
|
||||
worker)
|
||||
start_worker
|
||||
;;
|
||||
both)
|
||||
start_both
|
||||
;;
|
||||
*)
|
||||
echo "用法: $0 [api|worker|both]"
|
||||
echo ""
|
||||
echo " api - 启动 API 服务(默认)"
|
||||
echo " worker - 启动 Worker 服务"
|
||||
echo " both - 同时启动 API 和 Worker 服务"
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
}
|
||||
|
||||
main "$@"
|
||||
308
scripts/setup-env.sh
Executable file
308
scripts/setup-env.sh
Executable file
@@ -0,0 +1,308 @@
|
||||
#!/bin/bash
|
||||
# ============================================================================
|
||||
# 君鸿卡管系统 - 本地开发环境变量设置脚本
|
||||
# ============================================================================
|
||||
# 使用方法:
|
||||
# 1. 首次设置:./scripts/setup-env.sh
|
||||
# 2. 加载环境:source .env.local
|
||||
# 3. 启动服务:go run cmd/api/main.go
|
||||
# ============================================================================
|
||||
|
||||
set -e
|
||||
|
||||
# 颜色定义
|
||||
RED='\033[0;31m'
|
||||
GREEN='\033[0;32m'
|
||||
YELLOW='\033[1;33m'
|
||||
BLUE='\033[0;34m'
|
||||
CYAN='\033[0;36m'
|
||||
NC='\033[0m' # No Color
|
||||
|
||||
# 配置文件路径
|
||||
ENV_FILE=".env.local"
|
||||
|
||||
# 打印带颜色的消息
|
||||
print_header() {
|
||||
echo -e "\n${BLUE}========================================${NC}"
|
||||
echo -e "${BLUE} $1${NC}"
|
||||
echo -e "${BLUE}========================================${NC}\n"
|
||||
}
|
||||
|
||||
print_info() {
|
||||
echo -e "${CYAN}[INFO]${NC} $1"
|
||||
}
|
||||
|
||||
print_success() {
|
||||
echo -e "${GREEN}[OK]${NC} $1"
|
||||
}
|
||||
|
||||
print_warning() {
|
||||
echo -e "${YELLOW}[WARN]${NC} $1"
|
||||
}
|
||||
|
||||
print_error() {
|
||||
echo -e "${RED}[ERROR]${NC} $1"
|
||||
}
|
||||
|
||||
# 读取用户输入(带默认值)
|
||||
read_input() {
|
||||
local prompt="$1"
|
||||
local default="$2"
|
||||
local var_name="$3"
|
||||
local is_password="$4"
|
||||
|
||||
if [ "$is_password" = "true" ]; then
|
||||
echo -n -e "${CYAN}$prompt${NC} [默认: ****]: "
|
||||
read -s input
|
||||
echo ""
|
||||
else
|
||||
echo -n -e "${CYAN}$prompt${NC} [默认: $default]: "
|
||||
read input
|
||||
fi
|
||||
|
||||
if [ -z "$input" ]; then
|
||||
eval "$var_name='$default'"
|
||||
else
|
||||
eval "$var_name='$input'"
|
||||
fi
|
||||
}
|
||||
|
||||
# 生成随机密钥
|
||||
generate_secret() {
|
||||
if command -v openssl &> /dev/null; then
|
||||
openssl rand -hex 32
|
||||
else
|
||||
# 备用方案:使用 /dev/urandom
|
||||
head -c 32 /dev/urandom | xxd -p
|
||||
fi
|
||||
}
|
||||
|
||||
# 检查服务是否可用
|
||||
check_service() {
|
||||
local host="$1"
|
||||
local port="$2"
|
||||
local name="$3"
|
||||
|
||||
if command -v nc &> /dev/null; then
|
||||
if nc -z "$host" "$port" 2>/dev/null; then
|
||||
print_success "$name ($host:$port) 连接正常"
|
||||
return 0
|
||||
else
|
||||
print_warning "$name ($host:$port) 无法连接"
|
||||
return 1
|
||||
fi
|
||||
else
|
||||
print_info "跳过 $name 连接检查(nc 命令不可用)"
|
||||
return 0
|
||||
fi
|
||||
}
|
||||
|
||||
# 主流程
|
||||
main() {
|
||||
print_header "君鸿卡管系统 - 环境变量配置向导"
|
||||
|
||||
echo "本脚本将帮助您配置本地开发所需的环境变量。"
|
||||
echo "配置将保存到 $ENV_FILE 文件中。"
|
||||
echo ""
|
||||
|
||||
# 检查是否已存在配置文件
|
||||
if [ -f "$ENV_FILE" ]; then
|
||||
print_warning "发现已存在的配置文件: $ENV_FILE"
|
||||
echo -n "是否覆盖?[y/N]: "
|
||||
read confirm
|
||||
if [ "$confirm" != "y" ] && [ "$confirm" != "Y" ]; then
|
||||
print_info "已取消。如需更新配置,请手动编辑 $ENV_FILE"
|
||||
exit 0
|
||||
fi
|
||||
fi
|
||||
|
||||
# ========================================
|
||||
# 数据库配置
|
||||
# ========================================
|
||||
print_header "数据库配置 (PostgreSQL)"
|
||||
|
||||
read_input "数据库主机地址" "localhost" DB_HOST
|
||||
read_input "数据库端口" "5432" DB_PORT
|
||||
read_input "数据库用户名" "postgres" DB_USER
|
||||
read_input "数据库密码" "postgres" DB_PASSWORD "true"
|
||||
read_input "数据库名称" "junhong_cmp_dev" DB_NAME
|
||||
read_input "SSL 模式" "disable" DB_SSLMODE
|
||||
|
||||
# 测试数据库连接
|
||||
echo ""
|
||||
check_service "$DB_HOST" "$DB_PORT" "PostgreSQL"
|
||||
|
||||
# ========================================
|
||||
# Redis 配置
|
||||
# ========================================
|
||||
print_header "Redis 配置"
|
||||
|
||||
read_input "Redis 主机地址" "localhost" REDIS_ADDRESS
|
||||
read_input "Redis 端口" "6379" REDIS_PORT
|
||||
read_input "Redis 密码(无密码直接回车)" "" REDIS_PASSWORD "true"
|
||||
read_input "Redis 数据库编号" "0" REDIS_DB
|
||||
|
||||
# 测试 Redis 连接
|
||||
echo ""
|
||||
check_service "$REDIS_ADDRESS" "$REDIS_PORT" "Redis"
|
||||
|
||||
# ========================================
|
||||
# JWT 配置
|
||||
# ========================================
|
||||
print_header "JWT 配置"
|
||||
|
||||
DEFAULT_SECRET=$(generate_secret)
|
||||
read_input "JWT 密钥(直接回车生成随机密钥)" "$DEFAULT_SECRET" JWT_SECRET_KEY "true"
|
||||
|
||||
# ========================================
|
||||
# 服务器配置
|
||||
# ========================================
|
||||
print_header "服务器配置"
|
||||
|
||||
read_input "服务监听地址" ":3000" SERVER_ADDRESS
|
||||
read_input "日志级别 (debug/info/warn/error)" "debug" LOGGING_LEVEL
|
||||
|
||||
# ========================================
|
||||
# 可选:对象存储配置
|
||||
# ========================================
|
||||
print_header "对象存储配置(可选)"
|
||||
|
||||
echo -n "是否配置对象存储?[y/N]: "
|
||||
read configure_storage
|
||||
|
||||
if [ "$configure_storage" = "y" ] || [ "$configure_storage" = "Y" ]; then
|
||||
read_input "S3 端点" "" STORAGE_S3_ENDPOINT
|
||||
read_input "S3 区域" "" STORAGE_S3_REGION
|
||||
read_input "S3 存储桶" "" STORAGE_S3_BUCKET
|
||||
read_input "S3 Access Key ID" "" STORAGE_S3_ACCESS_KEY_ID
|
||||
read_input "S3 Secret Access Key" "" STORAGE_S3_SECRET_ACCESS_KEY "true"
|
||||
STORAGE_CONFIGURED="true"
|
||||
else
|
||||
STORAGE_CONFIGURED="false"
|
||||
fi
|
||||
|
||||
# ========================================
|
||||
# 生成配置文件
|
||||
# ========================================
|
||||
print_header "生成配置文件"
|
||||
|
||||
cat > "$ENV_FILE" << EOF
|
||||
# ============================================================================
|
||||
# 君鸿卡管系统 - 本地开发环境变量
|
||||
# 生成时间: $(date '+%Y-%m-%d %H:%M:%S')
|
||||
# ============================================================================
|
||||
# 使用方法:
|
||||
# source .env.local && go run cmd/api/main.go
|
||||
# 或者:
|
||||
# ./scripts/run-local.sh
|
||||
# ============================================================================
|
||||
|
||||
# ----------------------------------------------------------------------------
|
||||
# 数据库配置(必填)
|
||||
# ----------------------------------------------------------------------------
|
||||
export JUNHONG_DATABASE_HOST="$DB_HOST"
|
||||
export JUNHONG_DATABASE_PORT="$DB_PORT"
|
||||
export JUNHONG_DATABASE_USER="$DB_USER"
|
||||
export JUNHONG_DATABASE_PASSWORD="$DB_PASSWORD"
|
||||
export JUNHONG_DATABASE_DBNAME="$DB_NAME"
|
||||
export JUNHONG_DATABASE_SSLMODE="$DB_SSLMODE"
|
||||
|
||||
# ----------------------------------------------------------------------------
|
||||
# Redis 配置(必填)
|
||||
# ----------------------------------------------------------------------------
|
||||
export JUNHONG_REDIS_ADDRESS="$REDIS_ADDRESS"
|
||||
export JUNHONG_REDIS_PORT="$REDIS_PORT"
|
||||
export JUNHONG_REDIS_PASSWORD="$REDIS_PASSWORD"
|
||||
export JUNHONG_REDIS_DB="$REDIS_DB"
|
||||
|
||||
# ----------------------------------------------------------------------------
|
||||
# JWT 配置(必填)
|
||||
# ----------------------------------------------------------------------------
|
||||
export JUNHONG_JWT_SECRET_KEY="$JWT_SECRET_KEY"
|
||||
|
||||
# ----------------------------------------------------------------------------
|
||||
# 服务器配置
|
||||
# ----------------------------------------------------------------------------
|
||||
export JUNHONG_SERVER_ADDRESS="$SERVER_ADDRESS"
|
||||
|
||||
# ----------------------------------------------------------------------------
|
||||
# 日志配置
|
||||
# ----------------------------------------------------------------------------
|
||||
export JUNHONG_LOGGING_LEVEL="$LOGGING_LEVEL"
|
||||
export JUNHONG_LOGGING_DEVELOPMENT="true"
|
||||
export JUNHONG_LOGGING_APP_LOG_FILENAME="logs/app.log"
|
||||
export JUNHONG_LOGGING_ACCESS_LOG_FILENAME="logs/access.log"
|
||||
|
||||
EOF
|
||||
|
||||
# 添加对象存储配置(如果配置了)
|
||||
if [ "$STORAGE_CONFIGURED" = "true" ]; then
|
||||
cat >> "$ENV_FILE" << EOF
|
||||
# ----------------------------------------------------------------------------
|
||||
# 对象存储配置
|
||||
# ----------------------------------------------------------------------------
|
||||
export JUNHONG_STORAGE_PROVIDER="s3"
|
||||
export JUNHONG_STORAGE_S3_ENDPOINT="$STORAGE_S3_ENDPOINT"
|
||||
export JUNHONG_STORAGE_S3_REGION="$STORAGE_S3_REGION"
|
||||
export JUNHONG_STORAGE_S3_BUCKET="$STORAGE_S3_BUCKET"
|
||||
export JUNHONG_STORAGE_S3_ACCESS_KEY_ID="$STORAGE_S3_ACCESS_KEY_ID"
|
||||
export JUNHONG_STORAGE_S3_SECRET_ACCESS_KEY="$STORAGE_S3_SECRET_ACCESS_KEY"
|
||||
export JUNHONG_STORAGE_S3_USE_SSL="false"
|
||||
export JUNHONG_STORAGE_S3_PATH_STYLE="true"
|
||||
|
||||
EOF
|
||||
fi
|
||||
|
||||
# 添加迁移工具兼容配置
|
||||
cat >> "$ENV_FILE" << EOF
|
||||
# ----------------------------------------------------------------------------
|
||||
# 迁移工具兼容配置(用于 scripts/migrate.sh)
|
||||
# ----------------------------------------------------------------------------
|
||||
export MIGRATIONS_DIR="migrations"
|
||||
export DB_HOST="$DB_HOST"
|
||||
export DB_PORT="$DB_PORT"
|
||||
export DB_USER="$DB_USER"
|
||||
export DB_PASSWORD="$DB_PASSWORD"
|
||||
export DB_NAME="$DB_NAME"
|
||||
export DB_SSLMODE="$DB_SSLMODE"
|
||||
EOF
|
||||
|
||||
print_success "配置文件已生成: $ENV_FILE"
|
||||
|
||||
# ========================================
|
||||
# 后续步骤提示
|
||||
# ========================================
|
||||
print_header "设置完成!"
|
||||
|
||||
echo -e "${GREEN}环境变量已配置完成!${NC}"
|
||||
echo ""
|
||||
echo "后续步骤:"
|
||||
echo ""
|
||||
echo " 1. 加载环境变量:"
|
||||
echo -e " ${CYAN}source .env.local${NC}"
|
||||
echo ""
|
||||
echo " 2. 运行数据库迁移(如果需要):"
|
||||
echo -e " ${CYAN}./scripts/migrate.sh up${NC}"
|
||||
echo ""
|
||||
echo " 3. 启动 API 服务:"
|
||||
echo -e " ${CYAN}go run cmd/api/main.go${NC}"
|
||||
echo ""
|
||||
echo " 4. 启动 Worker 服务(可选):"
|
||||
echo -e " ${CYAN}go run cmd/worker/main.go${NC}"
|
||||
echo ""
|
||||
echo " 或者使用快捷脚本:"
|
||||
echo -e " ${CYAN}./scripts/run-local.sh${NC}"
|
||||
echo ""
|
||||
print_warning "注意:.env.local 包含敏感信息,请勿提交到 Git!"
|
||||
|
||||
# 确保 .env.local 在 .gitignore 中
|
||||
if [ -f ".gitignore" ]; then
|
||||
if ! grep -q "^\.env\.local$" .gitignore; then
|
||||
echo ".env.local" >> .gitignore
|
||||
print_info "已将 .env.local 添加到 .gitignore"
|
||||
fi
|
||||
fi
|
||||
}
|
||||
|
||||
# 运行主流程
|
||||
main
|
||||
48
scripts/test_storage.go
Normal file
48
scripts/test_storage.go
Normal file
@@ -0,0 +1,48 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"fmt"
|
||||
"os"
|
||||
"time"
|
||||
|
||||
"github.com/break/junhong_cmp_fiber/pkg/config"
|
||||
"github.com/break/junhong_cmp_fiber/pkg/storage"
|
||||
)
|
||||
|
||||
func main() {
|
||||
cfg, err := config.Load()
|
||||
if err != nil {
|
||||
fmt.Printf("加载配置失败: %v\n", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
|
||||
provider, err := storage.NewS3Provider(&cfg.Storage)
|
||||
if err != nil {
|
||||
fmt.Printf("创建 S3 Provider 失败: %v\n", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
|
||||
ctx := context.Background()
|
||||
|
||||
testContent := "iccid,msisdn\n89860012345678901234,13800000001\n89860012345678901235,13800000002\n"
|
||||
testKey := fmt.Sprintf("test/verify-%d.csv", time.Now().Unix())
|
||||
|
||||
fmt.Printf("上传文件到: %s/%s\n", cfg.Storage.S3.Bucket, testKey)
|
||||
|
||||
if err := provider.Upload(ctx, testKey, bytes.NewReader([]byte(testContent)), "text/csv"); err != nil {
|
||||
fmt.Printf("上传失败: %v\n", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
|
||||
exists, err := provider.Exists(ctx, testKey)
|
||||
if err != nil {
|
||||
fmt.Printf("检查存在失败: %v\n", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
|
||||
fmt.Printf("上传完成,文件存在: %v\n", exists)
|
||||
fmt.Printf("\n请在联通云后台检查 bucket '%s' 下的文件: %s\n", cfg.Storage.S3.Bucket, testKey)
|
||||
fmt.Println("文件未删除,请手动验证后删除")
|
||||
}
|
||||
@@ -553,7 +553,7 @@ func startTestWorker(t *testing.T, db *gorm.DB, rdb *redis.Client, logger *zap.L
|
||||
}
|
||||
|
||||
workerServer := queue.NewServer(rdb, queueCfg, logger)
|
||||
taskHandler := queue.NewHandler(db, rdb, logger)
|
||||
taskHandler := queue.NewHandler(db, rdb, nil, logger)
|
||||
taskHandler.RegisterHandlers()
|
||||
|
||||
go func() {
|
||||
|
||||
Reference in New Issue
Block a user