# Tasks: 数据持久化与异步任务处理集成

**Feature**: 002-gorm-postgres-asynq  
**Input**: Design documents from `/specs/002-gorm-postgres-asynq/`  
**Prerequisites**: plan.md, spec.md, data-model.md, contracts/api.yaml, research.md, quickstart.md

**Organization**: Tasks are grouped by user story (US1: 数据存储与检索, US2: 异步任务处理, US3: 连接管理与监控) to enable independent implementation and testing.

## Format: `[ID] [P?] [Story] Description`

- **[P]**: Can run in parallel (different files, no dependencies)
- **[Story]**: Which user story this task belongs to (US1, US2, US3)
- Include exact file paths in descriptions

---

## Phase 1: Setup (Shared Infrastructure)

**Purpose**: Project initialization and basic structure (project already exists, validate/enhance)

- [ ] T001 Validate project structure matches plan.md (internal/, pkg/, cmd/, configs/, migrations/, tests/)
- [ ] T002 Validate Go dependencies for Fiber + GORM + Asynq + Viper + Zap + golang-migrate
- [ ] T003 [P] Validate unified error codes in pkg/errors/codes.go and pkg/errors/errors.go
- [ ] T004 [P] Validate unified API response in pkg/response/response.go
- [ ] T005 [P] Add database configuration constants in pkg/constants/constants.go (DefaultMaxOpenConns=25, DefaultMaxIdleConns=10, etc.)
- [ ] T006 [P] Add task queue constants in pkg/constants/constants.go (TaskTypeEmailSend, TaskTypeDataSync, QueueCritical, QueueDefault, etc.)
- [ ] T007 [P] Add user/order status constants in pkg/constants/constants.go (UserStatusActive, OrderStatusPending, etc.)
- [ ] T008 [P] Add Redis key generation functions in pkg/constants/redis.go (RedisTaskLockKey, RedisTaskStatusKey)

---

## Phase 2: Foundational (Blocking Prerequisites)

**Purpose**: Core infrastructure that MUST be complete before ANY user story can be implemented

**⚠️ CRITICAL**: No user story work can begin until this phase is complete

- [ ] T009 Implement PostgreSQL connection initialization in pkg/database/postgres.go (GORM + connection pool)
- [ ] T010 Validate Redis connection initialization in pkg/database/redis.go (connection pool: PoolSize=10, MinIdleConns=5)
- [ ] T011 [P] Add DatabaseConfig to pkg/config/config.go (Host, Port, User, Password, MaxOpenConns, MaxIdleConns, ConnMaxLifetime)
- [ ] T012 [P] Add QueueConfig to pkg/config/config.go (Concurrency, Queues, RetryMax, Timeout)
- [ ] T013 [P] Update config.yaml files with database and queue configurations (config.dev.yaml, config.staging.yaml, config.prod.yaml)
- [ ] T014 Implement Asynq client initialization in pkg/queue/client.go (EnqueueTask with logging)
- [ ] T015 Implement Asynq server initialization in pkg/queue/server.go (with queue priorities and error handler)
- [ ] T016 Create base Store structure in internal/store/store.go with transaction support
- [ ] T017 Initialize postgres store in internal/store/postgres/store.go (embed UserStore, OrderStore)
- [ ] T018 Validate migrations directory structure (migrations/000001_init_schema.up.sql and .down.sql exist)

**Checkpoint**: Foundation ready - user story implementation can now begin in parallel

---

## Phase 3: User Story 1 - 可靠的数据存储与检索 (Priority: P1) 🎯 MVP

**Goal**: 实现可靠的数据持久化存储和高效的 CRUD 操作，确保数据一致性和完整性

**Independent Test**: 通过创建、读取、更新、删除用户和订单数据验证。包括基本 CRUD、事务提交、数据一致性验证等场景。

### Tests for User Story 1 (REQUIRED per Constitution)

> **NOTE: Write these tests FIRST, ensure they FAIL before implementation**

- [ ] T019 [P] [US1] Unit tests for User Store layer in tests/unit/store_test.go (Create, GetByID, Update, Delete, List)
- [ ] T020 [P] [US1] Unit tests for Order Store layer in tests/unit/store_test.go (Create, GetByID, Update, Delete, ListByUserID)
- [ ] T021 [P] [US1] Unit tests for User Service layer in tests/unit/service_test.go (business logic validation)
- [ ] T022 [P] [US1] Integration tests for User API endpoints in tests/integration/database_test.go (POST/GET/PUT/DELETE /users)
- [ ] T023 [P] [US1] Transaction rollback tests in tests/unit/store_test.go (verify atomic operations)

### Implementation for User Story 1

**Models & DTOs**:

- [ ] T024 [P] [US1] Validate BaseModel in internal/model/base.go (ID, CreatedAt, UpdatedAt, DeletedAt)
- [ ] T025 [P] [US1] Validate User model in internal/model/user.go with GORM tags (Username, Email, Password, Status)
- [ ] T026 [P] [US1] Validate Order model in internal/model/order.go with GORM tags (OrderID, UserID, Amount, Status)
- [ ] T027 [P] [US1] Validate User DTOs in internal/model/user_dto.go (CreateUserRequest, UpdateUserRequest, UserResponse, ListUsersResponse)
- [ ] T028 [P] [US1] Create Order DTOs in internal/model/order_dto.go (CreateOrderRequest, UpdateOrderRequest, OrderResponse, ListOrdersResponse)

**Store Layer (Data Access)**:

- [ ] T029 [US1] Implement UserStore in internal/store/postgres/user_store.go (Create, GetByID, Update, Delete, List with pagination)
- [ ] T030 [US1] Implement OrderStore in internal/store/postgres/order_store.go (Create, GetByID, Update, Delete, ListByUserID)
- [ ] T031 [US1] Add context timeout handling (3s default) and slow query logging (>100ms) in Store methods

**Service Layer (Business Logic)**:

- [ ] T032 [US1] Implement UserService in internal/service/user/service.go (CreateUser, GetUserByID, UpdateUser, DeleteUser, ListUsers)
- [ ] T033 [US1] Implement OrderService in internal/service/order/service.go (CreateOrder, GetOrderByID, UpdateOrder, DeleteOrder, ListOrdersByUserID)
- [ ] T034 [US1] Add password hashing (bcrypt) in UserService.CreateUser
- [ ] T035 [US1] Add validation logic in Service layer using Validator
- [ ] T036 [US1] Implement transaction example in OrderService (CreateOrderWithUser)

**Handler Layer (HTTP Endpoints)**:

- [ ] T037 [US1] Validate/enhance User Handler in internal/handler/user.go (Create, GetByID, Update, Delete, List endpoints)
- [ ] T038 [US1] Create Order Handler in internal/handler/order.go (Create, GetByID, Update, Delete, List endpoints)
- [ ] T039 [US1] Add request validation using Validator in handlers
- [ ] T040 [US1] Add unified error handling using pkg/errors/ and pkg/response/ in handlers
- [ ] T041 [US1] Add structured logging with Zap in handlers (log user_id, order_id, operation, duration)
- [ ] T042 [US1] Register User routes in cmd/api/main.go (POST/GET/PUT/DELETE /api/v1/users, /api/v1/users/:id)
- [ ] T043 [US1] Register Order routes in cmd/api/main.go (POST/GET/PUT/DELETE /api/v1/orders, /api/v1/orders/:id)

**Database Migrations**:

- [ ] T044 [US1] Validate migration 000001_init_schema.up.sql (tb_user and tb_order tables with indexes)
- [ ] T045 [US1] Validate migration 000001_init_schema.down.sql (DROP tables)
- [ ] T046 [US1] Test migration up/down with scripts/migrate.sh

**Checkpoint**: At this point, User Story 1 should be fully functional and testable independently

---

## Phase 4: User Story 2 - 异步任务处理能力 (Priority: P2)

**Goal**: 实现耗时操作的后台异步执行，避免阻塞用户请求，提升系统响应速度

**Independent Test**: 提交耗时任务（如发送邮件），验证任务被成功加入队列，用户请求立即返回，后台 Worker 完成任务执行。

### Tests for User Story 2 (REQUIRED per Constitution)

- [ ] T047 [P] [US2] Unit tests for Email task handler in tests/unit/task_handler_test.go (HandleEmailSend idempotency)
- [ ] T048 [P] [US2] Unit tests for Sync task handler in tests/unit/task_handler_test.go (HandleDataSync idempotency)
- [ ] T049 [P] [US2] Integration tests for task submission in tests/integration/task_test.go (EnqueueEmailTask, EnqueueSyncTask)
- [ ] T050 [P] [US2] Integration tests for task queue in tests/integration/task_test.go (verify Worker processes tasks)

### Implementation for User Story 2

**Task Payloads**:

- [ ] T051 [P] [US2] Validate EmailPayload in internal/task/email.go (RequestID, To, Subject, Body, CC, Attachments)
- [ ] T052 [P] [US2] Validate DataSyncPayload in internal/task/sync.go (RequestID, SyncType, StartDate, EndDate, BatchSize)
- [ ] T053 [P] [US2] Create SIMStatusSyncPayload in internal/task/sim.go (RequestID, ICCIDs, ForceSync)

**Task Handlers (Worker)**:

- [ ] T054 [US2] Implement HandleEmailSend in internal/task/email.go (with Redis idempotency lock and retry)
- [ ] T055 [US2] Implement HandleDataSync in internal/task/sync.go (with idempotency and batch processing)
- [ ] T056 [US2] Implement HandleSIMStatusSync in internal/task/sim.go (with idempotency)
- [ ] T057 [US2] Add structured logging in task handlers (task_id, task_type, request_id, duration)
- [ ] T058 [US2] Add error handling and retry logic in task handlers (max 5 retries, exponential backoff)

**Service Integration**:

- [ ] T059 [US2] Implement EmailService in internal/service/email/service.go (SendWelcomeEmail, EnqueueEmailTask)
- [ ] T060 [US2] Implement SyncService in internal/service/sync/service.go (EnqueueDataSyncTask, EnqueueSIMStatusSyncTask)
- [ ] T061 [US2] Add Queue Client dependency injection in Service constructors

**Handler Layer (Task Submission)**:

- [ ] T062 [US2] Validate/enhance Task Handler in internal/handler/task.go (SubmitEmailTask, SubmitSyncTask endpoints)
- [ ] T063 [US2] Add request validation for task payloads in handler
- [ ] T064 [US2] Add priority queue selection logic (critical/default/low) in handler
- [ ] T065 [US2] Register task routes in cmd/api/main.go (POST /api/v1/tasks/email, POST /api/v1/tasks/sync)

**Worker Process**:

- [ ] T066 [US2] Validate Worker main in cmd/worker/main.go (initialize Server, register handlers, graceful shutdown)
- [ ] T067 [US2] Register task handlers in Worker (HandleEmailSend, HandleDataSync, HandleSIMStatusSync)
- [ ] T068 [US2] Add signal handling for graceful shutdown in Worker (SIGINT, SIGTERM)

**Checkpoint**: At this point, User Stories 1 AND 2 should both work independently

---

## Phase 5: User Story 3 - 数据库连接管理与监控 (Priority: P3)

**Goal**: 监控数据库连接状态、查询性能和任务队列健康度，确保系统稳定运行

**Independent Test**: 通过健康检查接口验证数据库和 Redis 连接状态，模拟连接失败场景验证容错能力。

### Tests for User Story 3 (REQUIRED per Constitution)

- [ ] T069 [P] [US3] Integration tests for health check in tests/integration/health_test.go (GET /health returns 200 when healthy)
- [ ] T070 [P] [US3] Integration tests for degraded state in tests/integration/health_test.go (503 when database down)
- [ ] T071 [P] [US3] Unit tests for graceful shutdown in tests/unit/shutdown_test.go (verify connections closed)

### Implementation for User Story 3

**Health Check**:

- [ ] T072 [US3] Validate/enhance Health Handler in internal/handler/health.go (check PostgreSQL and Redis status)
- [ ] T073 [US3] Add database Ping check with timeout in Health Handler
- [ ] T074 [US3] Add Redis Ping check with timeout in Health Handler
- [ ] T075 [US3] Return appropriate status codes (200 ok, 503 degraded/unavailable)
- [ ] T076 [US3] Register health route in cmd/api/main.go (GET /health)

**Connection Management**:

- [ ] T077 [US3] Add connection pool monitoring in pkg/database/postgres.go (log Stats: OpenConnections, InUse, Idle)
- [ ] T078 [US3] Add connection retry logic in pkg/database/postgres.go (max 5 retries, exponential backoff)
- [ ] T079 [US3] Add slow query logging middleware in pkg/logger/middleware.go (log queries >100ms)

**Graceful Shutdown**:

- [ ] T080 [US3] Implement graceful shutdown in cmd/api/main.go (close DB, Redis, wait for requests, max 30s timeout)
- [ ] T081 [US3] Validate graceful shutdown in cmd/worker/main.go (stop accepting tasks, wait for completion, max 30s)
- [ ] T082 [US3] Add signal handling (SIGINT, SIGTERM) in both API and Worker processes

**Checkpoint**: All user stories should now be independently functional

---

## Phase 6: Polish & Quality Gates

**Purpose**: Improvements that affect multiple user stories and final quality checks

### Documentation (Constitution Principle VII - REQUIRED)

- [ ] T083 [P] Create feature summary doc in docs/002-gorm-postgres-asynq/功能总结.md (Chinese filename and content)
- [ ] T084 [P] Create usage guide in docs/002-gorm-postgres-asynq/使用指南.md (Chinese filename and content)
- [ ] T085 [P] Create architecture doc in docs/002-gorm-postgres-asynq/架构说明.md (Chinese filename and content)
- [ ] T086 Update README.md with brief feature description (2-3 sentences in Chinese)

### Code Quality

- [ ] T087 Code cleanup: Remove unused imports, variables, and functions
- [ ] T088 Code refactoring: Extract duplicate logic into helper functions
- [ ] T089 Performance optimization: Add database indexes for common queries (username, email, order_id, user_id, status)
- [ ] T090 Performance testing: Verify API response time P95 < 200ms, P99 < 500ms
- [ ] T091 [P] Additional unit tests to reach 70%+ overall coverage, 90%+ for Service layer
- [ ] T092 Security audit: Verify no SQL injection (GORM uses prepared statements)
- [ ] T093 Security audit: Verify password storage uses bcrypt hashing
- [ ] T094 Security audit: Verify sensitive data not logged (passwords, tokens)
- [ ] T095 Run quickstart.md validation (test all curl examples work)

### Quality Gates (Constitution Compliance)

- [ ] T096 Quality Gate: Run `go test ./...` (all tests pass)
- [ ] T097 Quality Gate: Run `gofmt -l .` (no formatting issues)
- [ ] T098 Quality Gate: Run `go vet ./...` (no issues)
- [ ] T099 Quality Gate: Run `golangci-lint run` (no critical issues)
- [ ] T100 Quality Gate: Verify test coverage with `go test -cover ./...` (70%+ overall, 90%+ Service)
- [ ] T101 Quality Gate: Check no TODO/FIXME remains (or documented in GitHub issues)
- [ ] T102 Quality Gate: Verify database migrations work (up and down)
- [ ] T103 Quality Gate: Verify API documentation in contracts/api.yaml matches implementation
- [ ] T104 Quality Gate: Verify no hardcoded constants (all use pkg/constants/)
- [ ] T105 Quality Gate: Verify no duplicate hardcoded values (3+ identical literals must be constants)
- [ ] T106 Quality Gate: Verify defined constants are used (no duplicate hardcoding)
- [ ] T107 Quality Gate: Verify code comments use Chinese (implementation comments in Chinese)
- [ ] T108 Quality Gate: Verify log messages use Chinese (logger.Info/Warn/Error/Debug in Chinese)
- [ ] T109 Quality Gate: Verify error messages support Chinese (user-facing errors have Chinese text)
- [ ] T110 Quality Gate: Verify no Java-style patterns (no getter/setter, no I-prefix, no Impl-suffix)
- [ ] T111 Quality Gate: Verify Go naming conventions (UserID not userId, HTTPServer not HttpServer)
- [ ] T112 Quality Gate: Verify error handling is explicit (no panic/recover in business logic)
- [ ] T113 Quality Gate: Verify uses goroutines/channels for concurrency (not thread pools)
- [ ] T114 Quality Gate: Verify no ORM associations (foreignKey, belongsTo tags - use manual joins)
- [ ] T115 Quality Gate: Verify feature docs created in docs/002-gorm-postgres-asynq/ with Chinese filenames
- [ ] T116 Quality Gate: Verify summary doc content uses Chinese
- [ ] T117 Quality Gate: Verify README.md updated with brief description
- [ ] T118 Quality Gate: Verify ALL HTTP requests logged to access.log (via pkg/logger/Middleware())
- [ ] T119 Quality Gate: Verify access log includes request/response bodies (limited to 50KB)
- [ ] T120 Quality Gate: Verify no middleware bypasses logging (test auth failures, rate limits)
- [ ] T121 Quality Gate: Verify access log has all required fields (method, path, status, duration_ms, request_id, ip, user_agent, request_body, response_body)

---

## Dependencies & Execution Order

### Phase Dependencies

- **Setup (Phase 1)**: No dependencies - can start immediately
- **Foundational (Phase 2)**: Depends on Setup completion - BLOCKS all user stories
- **User Stories (Phase 3-5)**: All depend on Foundational phase completion
  - User Story 1 (P1): Can start after Foundational - No dependencies on other stories
  - User Story 2 (P2): Can start after Foundational - Independent (may integrate with US1 for examples)
  - User Story 3 (P3): Can start after Foundational - Independent
- **Polish (Phase 6)**: Depends on all user stories being complete

### User Story Independence

- **US1 (P1)**: Fully independent - can be tested and deployed alone (MVP)
- **US2 (P2)**: Fully independent - can be tested and deployed alone (may reference US1 models as examples)
- **US3 (P3)**: Fully independent - can be tested and deployed alone

### Within Each User Story

- Tests MUST be written and FAIL before implementation
- Models → Store → Service → Handler → Routes
- Core implementation before integration
- Story complete before moving to next priority

### Parallel Opportunities

**Phase 1 (Setup)**:
- T003, T004, T005, T006, T007, T008 can all run in parallel

**Phase 2 (Foundational)**:
- T011, T012, T013 (config) can run in parallel
- T014, T015 (queue client/server) can run in parallel after config

**Phase 3 (User Story 1)**:
- T019-T023 (all tests) can run in parallel
- T024-T028 (all models/DTOs) can run in parallel
- T029, T030 (Store implementations) can run in parallel after models
- T032, T033 (Service implementations) can run in parallel after Store

**Phase 4 (User Story 2)**:
- T047-T050 (all tests) can run in parallel
- T051-T053 (all payloads) can run in parallel
- T054-T056 (all task handlers) can run in parallel after payloads
- T059, T060 (Service implementations) can run in parallel after handlers

**Phase 5 (User Story 3)**:
- T069-T071 (all tests) can run in parallel
- T073, T074 (Ping checks) can run in parallel
- T077, T078, T079 (connection management) can run in parallel

**Phase 6 (Polish)**:
- T083-T085 (all docs) can run in parallel
- T096-T121 (quality gates) run sequentially but can be automated in CI

---

## Parallel Example: User Story 1

```bash
# Launch all tests together:
go test -v tests/unit/store_test.go &       # T019, T020
go test -v tests/unit/service_test.go &     # T021
go test -v tests/integration/database_test.go &  # T022
wait

# Launch all models together:
Task: "Validate User model in internal/model/user.go"   # T025
Task: "Validate Order model in internal/model/order.go" # T026
Task: "Validate User DTOs"                               # T027
Task: "Create Order DTOs"                                # T028

# Launch both Store implementations together:
Task: "Implement UserStore"  # T029
Task: "Implement OrderStore" # T030
```

---

## Implementation Strategy

### MVP First (User Story 1 Only)

1. Complete Phase 1: Setup (T001-T008)
2. Complete Phase 2: Foundational (T009-T018) - CRITICAL
3. Complete Phase 3: User Story 1 (T019-T046)
4. **STOP and VALIDATE**: Test CRUD operations independently
5. Deploy/demo if ready

### Incremental Delivery

1. Setup + Foundational → Foundation ready
2. Add User Story 1 → Test independently → Deploy/Demo (MVP! 🎯)
3. Add User Story 2 → Test independently → Deploy/Demo
4. Add User Story 3 → Test independently → Deploy/Demo
5. Polish → Final quality checks → Production ready

### Parallel Team Strategy

With multiple developers:

1. Team completes Setup (Phase 1) + Foundational (Phase 2) together
2. Once Foundational is done:
   - Developer A: User Story 1 (T019-T046)
   - Developer B: User Story 2 (T047-T068)
   - Developer C: User Story 3 (T069-T082)
3. Stories complete and integrate independently
4. Team reconvenes for Polish (Phase 6)

---

## Notes

- [P] tasks = different files, no dependencies, can run in parallel
- [Story] label (US1, US2, US3) maps task to specific user story for traceability
- Each user story is independently completable and testable
- Tests are written FIRST and should FAIL before implementation (TDD approach)
- Commit after each task or logical group
- Stop at any checkpoint to validate story independently
- Project structure already exists - tasks validate/enhance existing code where noted
- Avoid: vague tasks, same file conflicts, cross-story dependencies that break independence

---

## Task Count Summary

- **Total Tasks**: 121
- **Phase 1 (Setup)**: 8 tasks
- **Phase 2 (Foundational)**: 10 tasks
- **Phase 3 (User Story 1)**: 28 tasks (5 tests + 23 implementation)
- **Phase 4 (User Story 2)**: 22 tasks (4 tests + 18 implementation)
- **Phase 5 (User Story 3)**: 14 tasks (3 tests + 11 implementation)
- **Phase 6 (Polish)**: 39 tasks (4 docs + 35 quality gates)

**Parallel Opportunities**: ~40 tasks marked [P] can run in parallel within their phases

**Suggested MVP Scope**: Phase 1 + Phase 2 + Phase 3 (User Story 1) = 46 tasks