Files
junhong_cmp_fiber/specs/001-fiber-middleware-integration/tasks.md

510 lines
22 KiB
Markdown

# Tasks: Fiber Middleware Integration with Configuration Management
**Feature**: 001-fiber-middleware-integration
**Input**: Design documents from `/specs/001-fiber-middleware-integration/`
**Prerequisites**: plan.md, spec.md, research.md, data-model.md, contracts/api.yaml
**Tests**: Unit and integration tests are REQUIRED per constitution testing standards (70%+ overall, 90%+ core business)
**Organization**: Tasks are grouped by user story to enable independent implementation and testing of each story.
## Format: `- [ ] [ID] [P?] [Story?] Description`
- **[P]**: Can run in parallel (different files, no dependencies)
- **[Story]**: Which user story this task belongs to (e.g., US1, US2, US3)
- Include exact file paths in descriptions
---
## Phase 1: Setup (Shared Infrastructure)
**Purpose**: Project initialization and basic structure
- [ ] T001 Create directory structure: pkg/config/, pkg/logger/, pkg/response/, pkg/errors/, pkg/constants/, pkg/validator/, internal/middleware/, configs/, logs/
- [ ] T002 [P] Setup unified error codes and messages in pkg/errors/codes.go
- [ ] T003 [P] Setup custom error types in pkg/errors/errors.go
- [ ] T004 [P] Setup unified response structure in pkg/response/response.go
- [ ] T005 [P] Setup response code constants in pkg/response/codes.go
- [ ] T006 [P] Setup business constants in pkg/constants/constants.go
- [ ] T007 [P] Setup Redis key generation functions in pkg/constants/redis.go
---
## Phase 2: Foundational (Blocking Prerequisites)
**Purpose**: Core infrastructure that MUST be complete before ANY user story can be implemented
**⚠️ CRITICAL**: No user story work can begin until this phase is complete
### Configuration Management (US1 Foundation)
- [ ] T008 Create Config structures with Viper mapstructure tags in pkg/config/config.go
- [ ] T009 Implement config loading with validation in pkg/config/loader.go
- [ ] T010 Implement config hot reload with fsnotify in pkg/config/watcher.go
- [ ] T011 Create default configuration file in configs/config.yaml
- [ ] T012 [P] Create environment-specific configs: config.dev.yaml, config.staging.yaml, config.prod.yaml
### Logging Infrastructure (US2 Foundation)
- [ ] T013 Initialize Zap logger with JSON encoder in pkg/logger/logger.go
- [ ] T014 Setup Lumberjack rotation for app.log in pkg/logger/rotation.go
- [ ] T015 Setup Lumberjack rotation for access.log in pkg/logger/rotation.go
- [ ] T016 Create Fiber logger middleware adapter in pkg/logger/middleware.go
### Redis Connection (US6 Foundation)
- [ ] T017 Setup Redis client with connection pool configuration in pkg/validator/redis.go or pkg/database/redis.go
**Checkpoint**: Foundation ready - user story implementation can now begin in parallel
---
## Phase 3: User Story 1 - Configuration Hot Reload (Priority: P1) 🎯 MVP
**Goal**: Enable runtime configuration updates without service restart
**Independent Test**: Modify config.yaml while server runs, verify changes applied within 5 seconds without restart
### Unit Tests for User Story 1
- [ ] T018 [P] [US1] Unit test for config loading and validation in pkg/config/loader_test.go
- [ ] T019 [P] [US1] Unit test for config hot reload mechanism in pkg/config/watcher_test.go
- [ ] T020 [P] [US1] Test invalid config handling (malformed YAML, validation errors) in pkg/config/config_test.go
### Implementation for User Story 1
- [ ] T021 [US1] Implement atomic config pointer swap in pkg/config/config.go (sync/atomic usage)
- [ ] T022 [US1] Implement config change callback with validation in pkg/config/watcher.go
- [ ] T023 [US1] Add config reload logging with Zap in pkg/config/watcher.go
- [ ] T024 [US1] Integrate config watcher with context cancellation in cmd/api/main.go
- [ ] T025 [US1] Add graceful shutdown for config watcher in cmd/api/main.go
**Checkpoint**: Config hot reload should work independently - modify config and see changes applied
---
## Phase 4: User Story 2 - Structured Logging and Log Rotation (Priority: P1)
**Goal**: Production-ready JSON logging with automatic rotation and separate app/access logs
**Independent Test**: Generate logs, verify JSON format in app.log and access.log, trigger rotation by size
### Unit Tests for User Story 2
- [ ] T026 [P] [US2] Unit test for logger initialization in pkg/logger/logger_test.go
- [ ] T027 [P] [US2] Unit test for log rotation configuration in pkg/logger/rotation_test.go
- [ ] T028 [P] [US2] Test structured logging with fields in pkg/logger/logger_test.go
### Implementation for User Story 2
- [ ] T029 [P] [US2] Create appLogger instance with Lumberjack writer in pkg/logger/logger.go
- [ ] T030 [P] [US2] Create accessLogger instance with separate Lumberjack writer in pkg/logger/logger.go
- [ ] T031 [US2] Configure JSON encoder with RFC3339 timestamps in pkg/logger/logger.go
- [ ] T032 [US2] Export GetAppLogger() and GetAccessLogger() functions in pkg/logger/logger.go
- [ ] T033 [US2] Add logger.Sync() call in graceful shutdown in cmd/api/main.go
**Checkpoint**: Both app.log and access.log should exist with JSON entries, rotate at configured size
---
## Phase 5: User Story 3 - Unified API Response Format (Priority: P1)
**Goal**: Consistent response structure across all endpoints
**Independent Test**: Call any endpoint, verify response has {code, data, msg, timestamp} structure
### Unit Tests for User Story 3
- [ ] T034 [P] [US3] Unit test for Success() response helper in pkg/response/response_test.go
- [ ] T035 [P] [US3] Unit test for Error() response helper in pkg/response/response_test.go
- [ ] T036 [P] [US3] Test response serialization with sonic JSON in pkg/response/response_test.go
### Implementation for User Story 3
- [ ] T037 [P] [US3] Implement Success() helper function in pkg/response/response.go
- [ ] T038 [P] [US3] Implement Error() helper function in pkg/response/response.go
- [ ] T039 [P] [US3] Implement SuccessWithMessage() helper function in pkg/response/response.go
- [ ] T040 [US3] Configure Fiber to use sonic as JSON serializer in cmd/api/main.go
- [ ] T041 [US3] Create example health check endpoint using response helpers in internal/handler/health.go
- [ ] T042 [US3] Register health check route in cmd/api/main.go
**Checkpoint**: Health check endpoint returns unified response format with proper structure
---
## Phase 6: User Story 4 - Request Logging and Tracing (Priority: P2)
**Goal**: Unique UUID v4 request ID for every request with comprehensive access logging
**Independent Test**: Make requests, verify each has unique X-Request-ID header and appears in logs
### Integration Tests for User Story 4
- [ ] T043 [P] [US4] Integration test for requestid middleware (UUID v4 generation) in tests/integration/middleware_test.go
- [ ] T044 [P] [US4] Integration test for logger middleware (access log entries) in tests/integration/middleware_test.go
- [ ] T045 [P] [US4] Test request ID propagation through middleware chain in tests/integration/middleware_test.go
### Implementation for User Story 4
- [ ] T046 [P] [US4] Configure Fiber requestid middleware with google/uuid in cmd/api/main.go
- [ ] T047 [US4] Implement custom logger middleware writing to accessLogger in internal/middleware/logger.go
- [ ] T048 [US4] Add request ID to Fiber Locals in logger middleware in internal/middleware/logger.go
- [ ] T049 [US4] Add X-Request-ID response header in logger middleware in internal/middleware/logger.go
- [ ] T050 [US4] Log request details (method, path, status, duration, IP, user_agent) to access.log in internal/middleware/logger.go
- [ ] T051 [US4] Register requestid and logger middleware in correct order in cmd/api/main.go
**Checkpoint**: Every request should have unique UUID v4 in header and access.log, with full request details
---
## Phase 7: User Story 5 - Automatic Error Recovery (Priority: P2)
**Goal**: Recover from panics, log stack trace, return error response, continue serving requests
**Independent Test**: Trigger panic in handler, verify server doesn't crash, returns 500, logs stack trace
### Integration Tests for User Story 5
- [ ] T052 [P] [US5] Integration test for panic recovery in tests/integration/middleware_test.go
- [ ] T053 [P] [US5] Test panic logging with stack trace in tests/integration/middleware_test.go
- [ ] T054 [P] [US5] Test subsequent requests after panic recovery in tests/integration/middleware_test.go
### Implementation for User Story 5
- [ ] T055 [US5] Implement custom recover middleware with Zap logging in internal/middleware/recover.go
- [ ] T056 [US5] Add stack trace capture to recover middleware in internal/middleware/recover.go
- [ ] T057 [US5] Add request ID to panic logs in internal/middleware/recover.go
- [ ] T058 [US5] Return unified error response (500, code 1000) on panic in internal/middleware/recover.go
- [ ] T059 [US5] Register recover middleware as FIRST middleware in cmd/api/main.go
- [ ] T060 [US5] Create test panic endpoint for testing in internal/handler/test.go (optional, for quickstart validation)
**Checkpoint**: Panic in handler should be caught, logged with stack trace, return 500, server continues running
---
## Phase 8: User Story 6 - Token-Based Authentication (Priority: P2)
**Goal**: Validate authentication tokens against Redis, enforce access control
**Independent Test**: Request with valid/invalid/missing token, verify 200/401 responses with correct error codes
### Unit Tests for User Story 6
- [ ] T061 [P] [US6] Unit test for TokenValidator.Validate() with valid token in pkg/validator/token_test.go
- [ ] T062 [P] [US6] Unit test for expired/invalid token (redis.Nil) in pkg/validator/token_test.go
- [ ] T063 [P] [US6] Unit test for Redis unavailable (fail closed) in pkg/validator/token_test.go
- [ ] T064 [P] [US6] Unit test for context timeout in Redis operations in pkg/validator/token_test.go
### Integration Tests for User Story 6
- [ ] T065 [P] [US6] Integration test for keyauth middleware with valid token in tests/integration/auth_test.go
- [ ] T066 [P] [US6] Integration test for missing token (401, code 1001) in tests/integration/auth_test.go
- [ ] T067 [P] [US6] Integration test for invalid token (401, code 1002) in tests/integration/auth_test.go
- [ ] T068 [P] [US6] Integration test for Redis down (503, code 1004) in tests/integration/auth_test.go
### Implementation for User Story 6
- [ ] T069 [US6] Create TokenValidator struct with Redis client in pkg/validator/token.go
- [ ] T070 [US6] Implement TokenValidator.Validate() with Redis GET operation in pkg/validator/token.go
- [ ] T071 [US6] Add context timeout (50ms) for Redis operations in pkg/validator/token.go
- [ ] T072 [US6] Implement Redis availability check (Ping) with fail-closed behavior in pkg/validator/token.go
- [ ] T073 [US6] Implement custom keyauth middleware wrapper in internal/middleware/auth.go
- [ ] T074 [US6] Configure keyauth with header lookup "token" in internal/middleware/auth.go
- [ ] T075 [US6] Add validator callback to keyauth config in internal/middleware/auth.go
- [ ] T076 [US6] Store user_id in Fiber Locals after successful validation in internal/middleware/auth.go
- [ ] T077 [US6] Implement custom ErrorHandler mapping errors to response codes in internal/middleware/auth.go
- [ ] T078 [US6] Add auth failure logging with request ID in internal/middleware/auth.go
- [ ] T079 [US6] Register keyauth middleware after logger in cmd/api/main.go
- [ ] T080 [US6] Create protected example endpoint (/api/v1/users) in internal/handler/user.go
- [ ] T081 [US6] Register protected routes with middleware in cmd/api/main.go
**Checkpoint**: Protected endpoints require valid token, reject invalid/missing tokens with correct error codes
---
## Phase 9: User Story 7 - Rate Limiting Configuration (Priority: P3)
**Goal**: Provide IP-based rate limiting capability (disabled by default, easy to enable)
**Independent Test**: Enable limiter, exceed limit, verify 429 responses; disable and verify no limiting
### Integration Tests for User Story 7
- [ ] T082 [P] [US7] Integration test for rate limiter with limit exceeded (429, code 1003) in tests/integration/ratelimit_test.go
- [ ] T083 [P] [US7] Integration test for rate limit reset after window expiration in tests/integration/ratelimit_test.go
- [ ] T084 [P] [US7] Test per-IP rate limiting (different IPs have separate limits) in tests/integration/ratelimit_test.go
### Implementation for User Story 7
- [ ] T085 [US7] Implement rate limiter middleware wrapper (COMMENTED by default) in internal/middleware/ratelimit.go
- [ ] T086 [US7] Configure limiter with IP-based key generator (c.IP()) in internal/middleware/ratelimit.go
- [ ] T087 [US7] Configure limiter with config values (Max, Expiration) in internal/middleware/ratelimit.go
- [ ] T088 [US7] Add custom LimitReached handler returning unified error response in internal/middleware/ratelimit.go
- [ ] T089 [US7] Add commented middleware registration example in cmd/api/main.go
- [ ] T090 [US7] Document rate limiter usage in quickstart.md (how to enable, configure)
- [ ] T091 [US7] Add rate limiter configuration examples to config files
**Checkpoint**: Rate limiter can be enabled via config, blocks excess requests per IP, returns 429 with code 1003
---
## Phase 10: Polish & Quality Gates
**Purpose**: Final quality checks and cross-cutting improvements
### Documentation & Examples
- [ ] T092 [P] Update quickstart.md with actual file paths and final configuration
- [ ] T093 [P] Create example requests (curl commands) in quickstart.md for all scenarios
- [ ] T094 [P] Document middleware execution order in docs/ or README
- [ ] T095 [P] Add troubleshooting section to quickstart.md
### Code Quality
- [ ] T096 [P] Add Go doc comments to all exported functions and types
- [ ] T097 [P] Run gofmt on all Go files
- [ ] T098 [P] Run go vet and fix all issues
- [ ] T099 [P] Run golangci-lint and fix critical issues
- [ ] T100 Review all Redis key usage, ensure no hardcoded strings (use constants.RedisAuthTokenKey())
- [ ] T101 Review all error handling, ensure explicit returns (no panic abuse)
- [ ] T102 Review naming conventions (UserID not userId, HTTPServer not HttpServer)
- [ ] T103 Check for Java-style anti-patterns (no I-prefix, no Impl-suffix, no getters/setters)
### Testing & Coverage
- [ ] T104 Run all unit tests: go test ./pkg/...
- [ ] T105 Run all integration tests: go test ./tests/integration/...
- [ ] T106 Measure test coverage: go test -cover ./...
- [ ] T107 Verify core business logic coverage >= 90% (config, logger, validator)
- [ ] T108 Verify overall coverage >= 70%
### Security Audit
- [ ] T109 Review authentication fail-closed behavior (Redis unavailable = 503)
- [ ] T110 Review context timeouts on Redis operations
- [ ] T111 Check for command injection vulnerabilities
- [ ] T112 Verify no sensitive data in logs (tokens, passwords)
- [ ] T113 Review error messages (no sensitive information leakage)
### Performance Validation
- [ ] T114 Test middleware overhead < 5ms per request (load testing)
- [ ] T115 Verify log rotation doesn't block requests
- [ ] T116 Test config hot reload doesn't affect in-flight requests
- [ ] T117 Verify Redis connection pool handles load correctly
### Final Quality Gates
- [ ] T118 Quality Gate: All tests pass (go test ./...)
- [ ] T119 Quality Gate: No formatting issues (gofmt -l . returns empty)
- [ ] T120 Quality Gate: No vet issues (go vet ./...)
- [ ] T121 Quality Gate: Test coverage meets requirements (70%+ overall, 90%+ core)
- [ ] T122 Quality Gate: All TODOs/FIXMEs addressed or documented
- [ ] T123 Quality Gate: quickstart.md works end-to-end (manual validation)
- [ ] T124 Quality Gate: All middleware integrated and working together
- [ ] T125 Quality Gate: Graceful shutdown works correctly (no goroutine leaks)
- [ ] T126 Quality Gate: Constitution compliance verified (no violations)
---
## Dependencies & Execution Order
### Phase Dependencies
1. **Setup (Phase 1)**: No dependencies - start immediately
2. **Foundational (Phase 2)**: Depends on Setup (Phase 1) - BLOCKS all user stories
3. **User Stories (Phases 3-9)**: All depend on Foundational (Phase 2) completion
- US1, US2, US3 can proceed in parallel (independent)
- US4 depends on US2 (needs logger)
- US5 can proceed in parallel with others
- US6 depends on US3 (needs response format)
- US7 depends on US3 (needs response format)
4. **Polish (Phase 10)**: Depends on all desired user stories being complete
### User Story Dependencies
```
Foundational (Phase 2) - MUST COMPLETE FIRST
├─→ US1: Config Hot Reload (independent)
├─→ US2: Logging (independent)
├─→ US3: Response Format (independent)
├─→ US4: Request Tracing (depends on US2: logger)
├─→ US5: Error Recovery (independent)
├─→ US6: Authentication (depends on US3: response format)
└─→ US7: Rate Limiting (depends on US3: response format)
```
### Within Each User Story
- Tests MUST be written FIRST and FAIL before implementation
- Models/structures before services
- Services before middleware
- Middleware before registration in main.go
- Core implementation before integration
### Parallel Opportunities
**Phase 1 (Setup)**: All tasks T002-T007 marked [P] can run in parallel
**Phase 2 (Foundational)**:
- T012 (dev/staging/prod configs) can run in parallel with others
- T013-T016 (logging) can run in parallel as group
- These are independent file operations
**Phase 3-9 (User Stories)**:
- After Foundational completes, these can start in parallel:
- US1: T018-T025 (config hot reload)
- US2: T026-T033 (logging)
- US3: T034-T042 (response format)
- US5: T052-T060 (error recovery)
- After US2 completes:
- US4: T043-T051 (request tracing)
- After US3 completes:
- US6: T061-T081 (authentication)
- US7: T082-T091 (rate limiting)
**Phase 10 (Polish)**: Many tasks marked [P] can run in parallel (documentation, linting, etc.)
---
## Parallel Execution Examples
### Example 1: Setup Phase (All Parallel)
```bash
# Launch all setup tasks together (different files):
Task T002: "Setup unified error codes in pkg/errors/codes.go"
Task T003: "Setup custom error types in pkg/errors/errors.go"
Task T004: "Setup unified response structure in pkg/response/response.go"
Task T005: "Setup response code constants in pkg/response/codes.go"
Task T006: "Setup business constants in pkg/constants/constants.go"
Task T007: "Setup Redis key generation functions in pkg/constants/redis.go"
```
### Example 2: User Story Tests (Parallel within Story)
```bash
# US6 unit tests - all can run in parallel:
Task T061: "Unit test for TokenValidator with valid token"
Task T062: "Unit test for expired/invalid token"
Task T063: "Unit test for Redis unavailable"
Task T064: "Unit test for context timeout"
# US6 integration tests - all can run in parallel:
Task T065: "Integration test with valid token"
Task T066: "Integration test for missing token"
Task T067: "Integration test for invalid token"
Task T068: "Integration test for Redis down"
```
### Example 3: Independent User Stories (After Foundation)
```bash
# After Phase 2 completes, these can all start in parallel:
Agent 1: Work on US1 (Config Hot Reload) - T018 through T025
Agent 2: Work on US2 (Logging) - T026 through T033
Agent 3: Work on US3 (Response Format) - T034 through T042
Agent 4: Work on US5 (Error Recovery) - T052 through T060
```
---
## Implementation Strategy
### MVP First (Minimum Viable Product)
**Recommendation**: Complete Phases 1-5 (Setup + Foundation + US1 + US2 + US3) for MVP
This delivers:
- Configuration hot reload (US1)
- Production logging (US2)
- API response consistency (US3)
Then deploy and validate before adding authentication and other features.
### Incremental Delivery Roadmap
1. **Foundation** (Phases 1-2): ~2-3 days
- Setup + Foundational infrastructure
- **Deliverable**: Basic Fiber app with config, logging, response structure
2. **MVP** (Phases 3-5): ~2-3 days
- US1: Config hot reload
- US2: Logging with rotation
- US3: Unified responses
- **Deliverable**: Production-ready foundation with observability
3. **Observability+** (Phases 6-7): ~2-3 days
- US4: Request tracing
- US5: Error recovery
- **Deliverable**: Complete observability and fault tolerance
4. **Security** (Phase 8): ~2-3 days
- US6: Authentication
- **Deliverable**: Secured API with Redis token validation
5. **Production Hardening** (Phases 9-10): ~2-3 days
- US7: Rate limiting (optional)
- Polish and quality gates
- **Deliverable**: Production-ready system
**Total Estimated Time**: 10-15 days (single developer)
### Parallel Team Strategy
With 3 developers after Foundation completes:
- **Dev A**: US1 + US2 (Config + Logging) - Core infrastructure
- **Dev B**: US3 + US4 (Response + Tracing) - API foundation
- **Dev C**: US5 + US6 (Recovery + Auth) - Reliability + Security
Then converge for US7 and Polish.
**Estimated Time with Parallel Work**: 7-10 days
---
## Task Summary
- **Total Tasks**: 126
- **Setup Phase**: 7 tasks
- **Foundational Phase**: 10 tasks (BLOCKING)
- **User Story 1** (Config): 8 tasks (3 tests + 5 implementation)
- **User Story 2** (Logging): 8 tasks (3 tests + 5 implementation)
- **User Story 3** (Response): 9 tasks (3 tests + 6 implementation)
- **User Story 4** (Tracing): 9 tasks (3 tests + 6 implementation)
- **User Story 5** (Recovery): 9 tasks (3 tests + 6 implementation)
- **User Story 6** (Auth): 21 tasks (8 tests + 13 implementation)
- **User Story 7** (Rate Limit): 10 tasks (3 tests + 7 implementation)
- **Polish Phase**: 35 tasks (documentation, quality, security)
**Parallelizable Tasks**: 45 tasks marked [P]
**Test Coverage**:
- Unit tests: 23 tasks
- Integration tests: 18 tasks
- Total test tasks: 41 (32% of all tasks)
**Constitution Compliance**:
- ✅ All tasks follow Handler → Service → Store → Model pattern
- ✅ All Redis keys use generation functions (no hardcoded strings)
- ✅ All responses use unified format
- ✅ All error codes centralized
- ✅ Go idiomatic patterns (no Java-style anti-patterns)
- ✅ Explicit error handling throughout
- ✅ Context usage for timeouts and cancellation
---
## Next Steps
1. ✅ Tasks.md generated and validated
2. ⏳ Review task list with team
3. ⏳ Set up development environment (Go 1.25.1, Redis 7.x)
4. ⏳ Run `/speckit.implement` to begin implementation
5. ⏳ Start with Phase 1 (Setup) - all tasks can run in parallel
**Ready for Implementation**: Yes - all tasks are specific, have file paths, and follow constitution principles.