junhong_cmp_fiber/specs/001-fiber-middleware-integration/tasks.md

# Tasks: Fiber Middleware Integration with Configuration Management

**Feature**: 001-fiber-middleware-integration  
**Input**: Design documents from `/specs/001-fiber-middleware-integration/`  
**Prerequisites**: plan.md, spec.md, research.md, data-model.md, contracts/api.yaml

**Tests**: Unit and integration tests are REQUIRED per constitution testing standards (70%+ overall, 90%+ core business)

**Organization**: Tasks are grouped by user story to enable independent implementation and testing of each story.

## Format: `- [ ] [ID] [P?] [Story?] Description`

- **[P]**: Can run in parallel (different files, no dependencies)
- **[Story]**: Which user story this task belongs to (e.g., US1, US2, US3)
- Include exact file paths in descriptions

---

## Phase 1: Setup (Shared Infrastructure)

**Purpose**: Project initialization and basic structure

- [X] T001 Create directory structure: pkg/config/, pkg/logger/, pkg/response/, pkg/errors/, pkg/constants/, pkg/validator/, internal/middleware/, configs/, logs/
- [X] T002 [P] Setup unified error codes and messages in pkg/errors/codes.go
- [X] T003 [P] Setup custom error types in pkg/errors/errors.go
- [X] T004 [P] Setup unified response structure in pkg/response/response.go
- [X] T005 [P] Setup response code constants in pkg/response/codes.go
- [X] T006 [P] Setup business constants in pkg/constants/constants.go
- [X] T007 [P] Setup Redis key generation functions in pkg/constants/redis.go

---

## Phase 2: Foundational (Blocking Prerequisites)

**Purpose**: Core infrastructure that MUST be complete before ANY user story can be implemented

**⚠️ CRITICAL**: No user story work can begin until this phase is complete

### Configuration Management (US1 Foundation)

- [X] T008 Create Config structures with Viper mapstructure tags in pkg/config/config.go
- [X] T009 Implement config loading with validation in pkg/config/loader.go
- [X] T010 Implement config hot reload with fsnotify in pkg/config/watcher.go
- [X] T011 Create default configuration file in configs/config.yaml
- [X] T012 [P] Create environment-specific configs: config.dev.yaml, config.staging.yaml, config.prod.yaml
- [X] T012a [P] Unit test for environment-specific config loading (test APP_ENV variable loads correct config file) in pkg/config/loader_test.go

### Logging Infrastructure (US2 Foundation)

- [X] T013 Initialize Zap logger with JSON encoder in pkg/logger/logger.go
- [X] T014 Setup Lumberjack rotation for app.log in pkg/logger/logger.go (合并到 T013)
- [X] T015 Setup Lumberjack rotation for access.log in pkg/logger/logger.go (合并到 T013)
- [X] T016 Create Fiber logger middleware adapter in pkg/logger/middleware.go

### Redis Connection (US6 Foundation)

- [X] T017 Setup Redis client with connection pool configuration in pkg/validator/token.go (使用 go-redis 直接在 main.go 中初始化)

**Checkpoint**: Foundation ready - user story implementation can now begin in parallel

---

## Phase 3: User Story 1 - Configuration Hot Reload (Priority: P1) 🎯 MVP

**Goal**: Enable runtime configuration updates without service restart

**Independent Test**: Modify config.yaml while server runs, verify changes applied within 5 seconds without restart

### Unit Tests for User Story 1

- [X] T018 [P] [US1] Unit test for config loading and validation in pkg/config/loader_test.go
- [X] T019 [P] [US1] Unit test for config hot reload mechanism in pkg/config/watcher_test.go
- [X] T020 [P] [US1] Test invalid config handling (malformed YAML, validation errors) in pkg/config/config_test.go

### Implementation for User Story 1

- [X] T021 [US1] Implement atomic config pointer swap in pkg/config/config.go (sync/atomic usage)
- [X] T022 [US1] Implement config change callback with validation in pkg/config/watcher.go
- [X] T023 [US1] Add config reload logging with Zap in pkg/config/watcher.go
- [X] T024 [US1] Integrate config watcher with context cancellation in cmd/api/main.go
- [X] T025 [US1] Add graceful shutdown for config watcher in cmd/api/main.go

**Checkpoint**: Config hot reload should work independently - modify config and see changes applied

---

## Phase 4: User Story 2 - Structured Logging and Log Rotation (Priority: P1)

**Goal**: Production-ready JSON logging with automatic rotation and separate app/access logs

**Independent Test**: Generate logs, verify JSON format in app.log and access.log, trigger rotation by size

### Unit Tests for User Story 2

- [X] T026 [P] [US2] Unit test for logger initialization in pkg/logger/logger_test.go
- [X] T027 [P] [US2] Unit test for log rotation configuration in pkg/logger/rotation_test.go
- [X] T028 [P] [US2] Test structured logging with fields in pkg/logger/logger_test.go

### Implementation for User Story 2

- [X] T029 [P] [US2] Create appLogger instance with Lumberjack writer in pkg/logger/logger.go
- [X] T030 [P] [US2] Create accessLogger instance with separate Lumberjack writer in pkg/logger/logger.go
- [X] T031 [US2] Configure JSON encoder with RFC3339 timestamps in pkg/logger/logger.go
- [X] T032 [US2] Export GetAppLogger() and GetAccessLogger() functions in pkg/logger/logger.go
- [X] T033 [US2] Add logger.Sync() call in graceful shutdown in cmd/api/main.go

**Checkpoint**: Both app.log and access.log should exist with JSON entries, rotate at configured size

---

## Phase 5: User Story 3 - Unified API Response Format (Priority: P1)

**Goal**: Consistent response structure across all endpoints

**Independent Test**: Call any endpoint, verify response has {code, data, msg, timestamp} structure

### Unit Tests for User Story 3

- [X] T034 [P] [US3] Unit test for Success() response helper in pkg/response/response_test.go
- [X] T035 [P] [US3] Unit test for Error() response helper in pkg/response/response_test.go
- [X] T036 [P] [US3] Test response serialization with sonic JSON in pkg/response/response_test.go

### Implementation for User Story 3

- [X] T037 [P] [US3] Implement Success() helper function in pkg/response/response.go
- [X] T038 [P] [US3] Implement Error() helper function in pkg/response/response.go
- [X] T039 [P] [US3] Implement SuccessWithMessage() helper function in pkg/response/response.go
- [X] T040 [US3] Configure Fiber to use sonic as JSON serializer in cmd/api/main.go
- [X] T041 [US3] Create example health check endpoint using response helpers in internal/handler/health.go
- [X] T042 [US3] Register health check route in cmd/api/main.go

**Checkpoint**: Health check endpoint returns unified response format with proper structure

---

## Phase 6: User Story 4 - Request Logging and Tracing (Priority: P2)

**Goal**: Unique UUID v4 request ID for every request with comprehensive access logging

**Independent Test**: Make requests, verify each has unique X-Request-ID header and appears in logs

### Integration Tests for User Story 4

- [X] T043 [P] [US4] Integration test for requestid middleware (UUID v4 generation) in tests/integration/middleware_test.go
- [X] T044 [P] [US4] Integration test for logger middleware (access log entries) in tests/integration/middleware_test.go
- [X] T045 [P] [US4] Test request ID propagation through middleware chain in tests/integration/middleware_test.go

### Implementation for User Story 4

- [X] T046 [P] [US4] Configure Fiber requestid middleware with google/uuid in cmd/api/main.go
- [X] T047 [US4] Implement custom logger middleware writing to accessLogger in internal/middleware/logger.go
- [X] T048 [US4] Add request ID to Fiber Locals in logger middleware in internal/middleware/logger.go
- [X] T049 [US4] Add X-Request-ID response header in logger middleware in internal/middleware/logger.go
- [X] T050 [US4] Log request details (method, path, status, duration, IP, user_agent) to access.log in internal/middleware/logger.go
- [X] T051 [US4] Register requestid and logger middleware in correct order in cmd/api/main.go

**Checkpoint**: Every request should have unique UUID v4 in header and access.log, with full request details

---

## Phase 7: User Story 5 - Automatic Error Recovery (Priority: P2)

**Goal**: Recover from panics, log stack trace, return error response, continue serving requests

**Independent Test**: Trigger panic in handler, verify server doesn't crash, returns 500, logs stack trace

### Integration Tests for User Story 5

- [X] T052 [P] [US5] Integration test for panic recovery in tests/integration/recover_test.go
- [X] T053 [P] [US5] Test panic logging with stack trace in tests/integration/recover_test.go
- [X] T054 [P] [US5] Test subsequent requests after panic recovery in tests/integration/recover_test.go

### Implementation for User Story 5

- [X] T055 [US5] Implement custom recover middleware with Zap logging in internal/middleware/recover.go
- [X] T056 [US5] Add stack trace capture to recover middleware in internal/middleware/recover.go
- [X] T057 [US5] Add request ID to panic logs in internal/middleware/recover.go
- [X] T058 [US5] Return unified error response (500, code 1000) on panic in internal/middleware/recover.go
- [X] T059 [US5] Register recover middleware as FIRST middleware in cmd/api/main.go
- [ ] T060 [US5] Create test panic endpoint for testing in internal/handler/test.go (optional, for quickstart validation)

**Checkpoint**: Panic in handler should be caught, logged with stack trace, return 500, server continues running

---

## Phase 8: User Story 6 - Token-Based Authentication (Priority: P2)

**Goal**: Validate authentication tokens against Redis, enforce access control

**Independent Test**: Request with valid/invalid/missing token, verify 200/401 responses with correct error codes

### Unit Tests for User Story 6

- [X] T061 [P] [US6] Unit test for TokenValidator.Validate() with valid token in pkg/validator/token_test.go
- [X] T062 [P] [US6] Unit test for expired/invalid token (redis.Nil) in pkg/validator/token_test.go
- [X] T063 [P] [US6] Unit test for Redis unavailable (fail closed) in pkg/validator/token_test.go
- [X] T064 [P] [US6] Unit test for context timeout in Redis operations in pkg/validator/token_test.go

### Integration Tests for User Story 6

- [X] T065 [P] [US6] Integration test for keyauth middleware with valid token in tests/integration/auth_test.go
- [X] T066 [P] [US6] Integration test for missing token (401, code 1001) in tests/integration/auth_test.go
- [X] T067 [P] [US6] Integration test for invalid token (401, code 1002) in tests/integration/auth_test.go
- [X] T068 [P] [US6] Integration test for Redis down (503, code 1004) in tests/integration/auth_test.go

### Implementation for User Story 6

- [X] T069 [US6] Create TokenValidator struct with Redis client in pkg/validator/token.go
- [X] T070 [US6] Implement TokenValidator.Validate() with Redis GET operation in pkg/validator/token.go
- [X] T071 [US6] Add context timeout (50ms) for Redis operations in pkg/validator/token.go
- [X] T072 [US6] Implement Redis availability check (Ping) with fail-closed behavior in pkg/validator/token.go
- [X] T073 [US6] Implement custom keyauth middleware wrapper in internal/middleware/auth.go
- [X] T074 [US6] Configure keyauth with header lookup "token" in internal/middleware/auth.go
- [X] T075 [US6] Add validator callback to keyauth config in internal/middleware/auth.go
- [X] T076 [US6] Store user_id in Fiber Locals after successful validation in internal/middleware/auth.go
- [X] T077 [US6] Implement custom ErrorHandler mapping errors to response codes in internal/middleware/auth.go
- [X] T078 [US6] Add auth failure logging with request ID in internal/middleware/auth.go
- [X] T079 [US6] Register keyauth middleware after logger in cmd/api/main.go
- [X] T080 [US6] Create protected example endpoint (/api/v1/users) in internal/handler/user.go
- [X] T081 [US6] Register protected routes with middleware in cmd/api/main.go

**Checkpoint**: Protected endpoints require valid token, reject invalid/missing tokens with correct error codes

---

## Phase 9: User Story 7 - Rate Limiting Configuration (Priority: P3)

**Goal**: Provide IP-based rate limiting capability (disabled by default, easy to enable)

**Independent Test**: Enable limiter, exceed limit, verify 429 responses; disable and verify no limiting

### Integration Tests for User Story 7

- [X] T082 [P] [US7] Integration test for rate limiter with limit exceeded (429, code 1003) in tests/integration/ratelimit_test.go
- [X] T083 [P] [US7] Integration test for rate limit reset after window expiration in tests/integration/ratelimit_test.go
- [X] T084 [P] [US7] Test per-IP rate limiting (different IPs have separate limits) in tests/integration/ratelimit_test.go

### Implementation for User Story 7

- [X] T085 [US7] Implement rate limiter middleware wrapper (COMMENTED by default) in internal/middleware/ratelimit.go
- [X] T086 [US7] Configure limiter with IP-based key generator (c.IP()) in internal/middleware/ratelimit.go
- [X] T087 [US7] Configure limiter with config values (Max, Expiration) in internal/middleware/ratelimit.go
- [X] T088 [US7] Add custom LimitReached handler returning unified error response in internal/middleware/ratelimit.go
- [X] T089 [US7] Add commented middleware registration example in cmd/api/main.go
- [X] T090 [US7] Document rate limiter usage in quickstart.md (how to enable, configure)
- [X] T091 [US7] Add rate limiter configuration examples to config files

**Checkpoint**: Rate limiter can be enabled via config, blocks excess requests per IP, returns 429 with code 1003

---

## Phase 10: Polish & Quality Gates

**Purpose**: Final quality checks and cross-cutting improvements

### Documentation & Examples

- [X] T092 [P] Update quickstart.md with actual file paths and final configuration
- [X] T093 [P] Create example requests (curl commands) in quickstart.md for all scenarios
- [X] T094 [P] Document middleware execution order in docs/ or README
- [X] T095 [P] Add troubleshooting section to quickstart.md
- [X] T095a [P] Create docs/rate-limiting.md with configuration guide, code examples, testing instructions, storage options comparison, and common usage patterns (implements FR-020)

### Code Quality

- [X] T096 [P] Add Go doc comments to all exported functions and types
- [X] T097 [P] Run code quality checks (gofmt, go vet, golangci-lint) on all Go files
- [X] T098 [P] Fix all formatting, linting, and static analysis issues reported by T097
- [X] T099 Review all Redis key usage, ensure no hardcoded strings (use constants.RedisAuthTokenKey())
- [X] T101 Review all error handling, ensure explicit returns (no panic abuse)
- [X] T102 Review naming conventions (UserID not userId, HTTPServer not HttpServer)
- [X] T103 Check for Java-style anti-patterns (no I-prefix, no Impl-suffix, no getters/setters)

### Testing & Coverage

- [X] T104 Run all unit tests: go test ./pkg/...
- [X] T105 Run all integration tests: go test ./tests/integration/...
- [X] T106 Measure test coverage: go test -cover ./...
- [X] T107 Verify core business logic coverage >= 90% (config, logger, validator)
- [X] T108 Verify overall coverage >= 70%

### Security Audit

- [X] T109 Review authentication fail-closed behavior (Redis unavailable = 503)
- [X] T110 Review context timeouts on Redis operations
- [X] T111 Check for command injection vulnerabilities
- [X] T112 Verify no sensitive data in logs (tokens, passwords)
- [X] T113 Review error messages (no sensitive information leakage)

### Performance Validation

- [X] T114 Test middleware overhead < 5ms per request (load testing)
- [X] T115 Verify log rotation doesn't block requests
- [X] T116 Test config hot reload doesn't affect in-flight requests
- [X] T117 Verify Redis connection pool handles load correctly

### Final Quality Gates

- [X] T118 Quality Gate: All tests pass (go test ./...)
- [X] T119 Quality Gate: No formatting issues (gofmt -l . returns empty)
- [X] T120 Quality Gate: No vet issues (go vet ./...)
- [X] T121 Quality Gate: Test coverage meets requirements (70%+ overall, 90%+ core)
- [X] T122 Quality Gate: All TODOs/FIXMEs addressed or documented
- [X] T123 Quality Gate: quickstart.md works end-to-end (manual validation)
- [X] T124 Quality Gate: All middleware integrated and working together
- [X] T125 Quality Gate: Graceful shutdown works correctly (no goroutine leaks)
- [X] T126 Quality Gate: Constitution compliance verified (no violations)

---

## Dependencies & Execution Order

### Phase Dependencies

1. **Setup (Phase 1)**: No dependencies - start immediately
2. **Foundational (Phase 2)**: Depends on Setup (Phase 1) - BLOCKS all user stories
3. **User Stories (Phases 3-9)**: All depend on Foundational (Phase 2) completion
   - US1, US2, US3 can proceed in parallel (independent)
   - US4 depends on US2 (needs logger)
   - US5 can proceed in parallel with others
   - US6 depends on US3 (needs response format)
   - US7 depends on US3 (needs response format)
4. **Polish (Phase 10)**: Depends on all desired user stories being complete

### User Story Dependencies

```
Foundational (Phase 2) - MUST COMPLETE FIRST
    ├─→ US1: Config Hot Reload (independent)
    ├─→ US2: Logging (independent)
    ├─→ US3: Response Format (independent)
    │
    ├─→ US4: Request Tracing (depends on US2: logger)
    ├─→ US5: Error Recovery (independent)
    │
    ├─→ US6: Authentication (depends on US3: response format)
    └─→ US7: Rate Limiting (depends on US3: response format)
```

### Within Each User Story

- Tests MUST be written FIRST and FAIL before implementation
- Models/structures before services
- Services before middleware
- Middleware before registration in main.go
- Core implementation before integration

### Parallel Opportunities

**Phase 1 (Setup)**: All tasks T002-T007 marked [P] can run in parallel

**Phase 2 (Foundational)**:
- T012 (dev/staging/prod configs) can run in parallel with others
- T013-T016 (logging) can run in parallel as group
- These are independent file operations

**Phase 3-9 (User Stories)**:
- After Foundational completes, these can start in parallel:
  - US1: T018-T025 (config hot reload)
  - US2: T026-T033 (logging)
  - US3: T034-T042 (response format)
  - US5: T052-T060 (error recovery)
- After US2 completes:
  - US4: T043-T051 (request tracing)
- After US3 completes:
  - US6: T061-T081 (authentication)
  - US7: T082-T091 (rate limiting)

**Phase 10 (Polish)**: Many tasks marked [P] can run in parallel (documentation, linting, etc.)

---

## Parallel Execution Examples

### Example 1: Setup Phase (All Parallel)

```bash
# Launch all setup tasks together (different files):
Task T002: "Setup unified error codes in pkg/errors/codes.go"
Task T003: "Setup custom error types in pkg/errors/errors.go"
Task T004: "Setup unified response structure in pkg/response/response.go"
Task T005: "Setup response code constants in pkg/response/codes.go"
Task T006: "Setup business constants in pkg/constants/constants.go"
Task T007: "Setup Redis key generation functions in pkg/constants/redis.go"
```

### Example 2: User Story Tests (Parallel within Story)

```bash
# US6 unit tests - all can run in parallel:
Task T061: "Unit test for TokenValidator with valid token"
Task T062: "Unit test for expired/invalid token"
Task T063: "Unit test for Redis unavailable"
Task T064: "Unit test for context timeout"

# US6 integration tests - all can run in parallel:
Task T065: "Integration test with valid token"
Task T066: "Integration test for missing token"
Task T067: "Integration test for invalid token"
Task T068: "Integration test for Redis down"
```

### Example 3: Independent User Stories (After Foundation)

```bash
# After Phase 2 completes, these can all start in parallel:
Agent 1: Work on US1 (Config Hot Reload) - T018 through T025
Agent 2: Work on US2 (Logging) - T026 through T033
Agent 3: Work on US3 (Response Format) - T034 through T042
Agent 4: Work on US5 (Error Recovery) - T052 through T060
```

---

## Implementation Strategy

### MVP First (Minimum Viable Product)

**Recommendation**: Complete Phases 1-5 (Setup + Foundation + US1 + US2 + US3) for MVP

This delivers:
- Configuration hot reload (US1)
- Production logging (US2)
- API response consistency (US3)

Then deploy and validate before adding authentication and other features.

### Incremental Delivery Roadmap

1. **Foundation** (Phases 1-2): ~2-3 days
   - Setup + Foundational infrastructure
   - **Deliverable**: Basic Fiber app with config, logging, response structure

2. **MVP** (Phases 3-5): ~2-3 days
   - US1: Config hot reload
   - US2: Logging with rotation
   - US3: Unified responses
   - **Deliverable**: Production-ready foundation with observability

3. **Observability+** (Phases 6-7): ~2-3 days
   - US4: Request tracing
   - US5: Error recovery
   - **Deliverable**: Complete observability and fault tolerance

4. **Security** (Phase 8): ~2-3 days
   - US6: Authentication
   - **Deliverable**: Secured API with Redis token validation

5. **Production Hardening** (Phases 9-10): ~2-3 days
   - US7: Rate limiting (optional)
   - Polish and quality gates
   - **Deliverable**: Production-ready system

**Total Estimated Time**: 10-15 days (single developer)

### Parallel Team Strategy

With 3 developers after Foundation completes:

- **Dev A**: US1 + US2 (Config + Logging) - Core infrastructure
- **Dev B**: US3 + US4 (Response + Tracing) - API foundation
- **Dev C**: US5 + US6 (Recovery + Auth) - Reliability + Security

Then converge for US7 and Polish.

**Estimated Time with Parallel Work**: 7-10 days

---

## Task Summary

- **Total Tasks**: 127 (updated: added T012a for environment config testing, T095a for rate-limiting.md documentation, consolidated code quality checks from 4 tasks into 3 tasks)
- **Setup Phase**: 7 tasks
- **Foundational Phase**: 11 tasks (BLOCKING) - includes T012a
- **User Story 1** (Config): 8 tasks (3 tests + 5 implementation)
- **User Story 2** (Logging): 8 tasks (3 tests + 5 implementation)
- **User Story 3** (Response): 9 tasks (3 tests + 6 implementation)
- **User Story 4** (Tracing): 9 tasks (3 tests + 6 implementation)
- **User Story 5** (Recovery): 9 tasks (3 tests + 6 implementation)
- **User Story 6** (Auth): 21 tasks (8 tests + 13 implementation)
- **User Story 7** (Rate Limit): 10 tasks (3 tests + 7 implementation)
- **Polish Phase**: 35 tasks (documentation, quality, security) - includes T095a, consolidated code quality checks

**Parallelizable Tasks**: 47 tasks marked [P] (added T012a, T095a)

**Test Coverage**:
- Unit tests: 24 tasks (added T012a)
- Integration tests: 18 tasks
- Total test tasks: 42 (33% of all tasks)

**Constitution Compliance**:
- ✅ All tasks follow Handler → Service → Store → Model pattern
- ✅ All Redis keys use generation functions (no hardcoded strings)
- ✅ All responses use unified format
- ✅ All error codes centralized
- ✅ Go idiomatic patterns (no Java-style anti-patterns)
- ✅ Explicit error handling throughout
- ✅ Context usage for timeouts and cancellation

---

## Next Steps

1. ✅ Tasks.md generated and validated
2. ⏳ Review task list with team
3. ⏳ Set up development environment (Go 1.25.1, Redis 7.x)
4. ⏳ Run `/speckit.implement` to begin implementation
5. ⏳ Start with Phase 1 (Setup) - all tasks can run in parallel

**Ready for Implementation**: Yes - all tasks are specific, have file paths, and follow constitution principles.