做完了一部分,备份一下,防止以外删除

This commit is contained in:
2025-11-11 15:16:38 +08:00
parent 9600e5b6e0
commit e98dd4d725
39 changed files with 2423 additions and 183 deletions

View File

@@ -1,7 +1,7 @@
# Data Model: Fiber Middleware Integration
**Feature**: 001-fiber-middleware-integration
**Date**: 2025-11-10
**Feature**: 001-fiber-middleware-integration
**Date**: 2025-11-10
**Phase**: 1 - Design & Contracts
## Overview
@@ -162,9 +162,9 @@ middleware:
### AuthToken Entity
**Storage**: Redis key-value pair
**Key Format**: `auth:token:{token_string}` (generated via `constants.RedisAuthTokenKey()`)
**Value Format**: Plain string containing user ID
**Storage**: Redis key-value pair
**Key Format**: `auth:token:{token_string}` (generated via `constants.RedisAuthTokenKey()`)
**Value Format**: Plain string containing user ID
**TTL**: Managed by Redis (set when token is created)
```go
@@ -218,16 +218,16 @@ func RedisAuthTokenKey(token string) string {
// Access via: c.Locals("key")
// Request ID (set by requestid middleware)
requestID := c.Locals("requestid").(string) // UUID v4 string
requestID := c.Locals(constants.ContextKeyRequestID).(string) // UUID v4 string
// User ID (set by keyauth middleware after validation)
userID, ok := c.Locals("user_id").(string)
userID, ok := c.Locals(constants.ContextKeyUserID).(string)
if !ok {
// Not authenticated
}
// Start time (for duration calculation)
startTime := c.Locals("start_time").(time.Time)
startTime := c.Locals(constants.ContextKeyStartTime).(time.Time)
```
**Context Keys** (constants in `pkg/constants/constants.go`):
@@ -254,8 +254,8 @@ const (
### Application Log Entry
**Format**: JSON
**Output**: `logs/app.log`
**Format**: JSON
**Output**: `logs/app.log`
**Logger**: Zap (appLogger instance)
```json
@@ -287,17 +287,21 @@ const (
```go
appLogger.Info("User created successfully",
zap.String("request_id", requestID),
zap.String("user_id", userID),
zap.String(constants.ContextKeyUserID, userID),
zap.String("username", username),
zap.String("ip", ip),
)
```
### Access Log Entry
### Access Log Entry Format
**Format**: JSON
**Output**: `logs/access.log`
**Purpose**: Records all HTTP requests for audit trail, performance monitoring, and troubleshooting
**Format**: JSON (one entry per line)
**Output**: `logs/access.log`
**Logger**: Zap (accessLogger instance)
**Requirement**: Implements spec.md FR-011
**Complete JSON Schema**:
```json
{
@@ -309,22 +313,32 @@ appLogger.Info("User created successfully",
"duration_ms": 45.234,
"request_id": "550e8400-e29b-41d4-a716-446655440000",
"ip": "192.168.1.100",
"user_agent": "Mozilla/5.0...",
"user_agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7)...",
"user_id": "user-789"
}
```
**Fields**:
- `timestamp`: ISO 8601 format (RFC3339)
- `level`: Always "info" for access logs
- `method`: HTTP method (GET, POST, PUT, DELETE)
- `path`: Request path
- `status`: HTTP status code
- `duration_ms`: Request duration in milliseconds
- `request_id`: Request correlation ID
- `ip`: Client IP address
- `user_agent`: User-Agent header
- `user_id`: Authenticated user ID (if available, empty if not)
**Field Definitions**:
| Field | Type | Required | Description | Example |
|-------|------|----------|-------------|---------|
| `timestamp` | string | Yes | ISO 8601 timestamp (RFC3339 with milliseconds) | `2025-11-10T15:30:45.123Z` |
| `level` | string | Yes | Log level (always "info" for access logs) | `info` |
| `method` | string | Yes | HTTP method | `GET`, `POST`, `PUT`, `DELETE`, `PATCH` |
| `path` | string | Yes | Request path (including query params) | `/api/v1/users?page=1` |
| `status` | int | Yes | HTTP status code | `200`, `401`, `500` |
| `duration_ms` | float64 | Yes | Request processing duration in milliseconds | `45.234` |
| `request_id` | string | Yes | UUID v4 request identifier | `550e8400-e29b-41d4-a716-446655440000` |
| `ip` | string | Yes | Client IP address | `192.168.1.100` |
| `user_agent` | string | Yes | User-Agent header value | `Mozilla/5.0...` |
| `user_id` | string | No | Authenticated user ID (empty string if not authenticated) | `user-789` or `""` |
**Notes**:
- All access logs are written at "info" level
- `user_id` field is empty string (`""`) for unauthenticated requests
- `duration_ms` includes full middleware chain execution time
- `path` includes query parameters for complete request tracking
- Logged after response is sent (includes actual status code)
**Zap Usage**:
@@ -337,7 +351,7 @@ accessLogger.Info("",
zap.String("request_id", requestID),
zap.String("ip", c.IP()),
zap.String("user_agent", c.Get("User-Agent")),
zap.String("user_id", userID),
zap.String(constants.ContextKeyUserID, userID),
)
```
@@ -531,7 +545,7 @@ func Wrap(code int, message string, err error) *AppError {
### Rate Limit Tracking
**Storage**: In-memory (default) or Redis (for distributed)
**Storage**: In-memory (default) or Redis (for distributed)
**Managed by**: Fiber limiter middleware (internal storage)
```go
@@ -625,16 +639,16 @@ func RedisRateLimitKey(ip string) string {
2. Recover Middleware (catch panics)
3. RequestID Middleware (generate UUID v4)
→ Store in c.Locals("requestid")
→ Store in c.Locals(constants.ContextKeyRequestID)
4. Logger Middleware (start)
→ Store start_time in c.Locals("start_time")
→ Store start_time in c.Locals(constants.ContextKeyStartTime)
5. KeyAuth Middleware
→ Extract token from header
→ Call TokenValidator.Validate(token)
→ Validate with Redis: GET auth:token:{token}
→ If valid: Store user_id in c.Locals("user_id")
→ If valid: Store user_id in c.Locals(constants.ContextKeyUserID)
→ If invalid: Return 401 with error code
→ If Redis down: Return 503 with error code
@@ -643,7 +657,7 @@ func RedisRateLimitKey(ip string) string {
→ If exceeded: Return 429 with error code
7. Handler (business logic)
→ Access c.Locals("requestid"), c.Locals("user_id")
→ Access c.Locals(constants.ContextKeyRequestID), c.Locals(constants.ContextKeyUserID)
→ Process request
→ Return response via response.Success() or response.Error()
@@ -676,3 +690,142 @@ func RedisRateLimitKey(ip string) string {
- ✅ No Java-style patterns (no I-prefix, no Impl-suffix)
**Next**: Generate API contracts (OpenAPI specification)
---
## 8. Configuration Validation Rules
This section defines comprehensive validation constraints for all configuration fields, implementing the requirements in spec.md FR-003.
### Validation Error Format
All validation errors MUST follow this format:
```
"Invalid configuration: {field_path}: {error_reason} (current value: {value}, expected: {constraint})"
```
Example:
```
"Invalid configuration: server.read_timeout: duration out of range (current value: 1s, expected: 5s-300s)"
```
---
### Server Configuration Validation
| Field | Type | Required | Constraint | Default | Error Message |
|-------|------|----------|------------|---------|---------------|
| `server.address` | string | Yes | Non-empty, format `:PORT` or `HOST:PORT` | `:3000` | "server.address: must be non-empty and in format ':PORT' or 'HOST:PORT'" |
| `server.read_timeout` | duration | Yes | 5s - 300s | `10s` | "server.read_timeout: duration out of range (expected: 5s-300s)" |
| `server.write_timeout` | duration | Yes | 5s - 300s | `10s` | "server.write_timeout: duration out of range (expected: 5s-300s)" |
| `server.shutdown_timeout` | duration | Yes | 10s - 120s | `30s` | "server.shutdown_timeout: duration out of range (expected: 10s-120s)" |
| `server.prefork` | bool | No | true or false | `false` | "server.prefork: must be boolean (true/false)" |
---
### Redis Configuration Validation
| Field | Type | Required | Constraint | Default | Error Message |
|-------|------|----------|------------|---------|---------------|
| `redis.address` | string | Yes | Non-empty, format `HOST:PORT` | `localhost:6379` | "redis.address: must be non-empty and in format 'HOST:PORT'" |
| `redis.password` | string | No | Any string (empty allowed) | `""` | N/A |
| `redis.db` | int | No | 0 - 15 | `0` | "redis.db: database number out of range (expected: 0-15)" |
| `redis.pool_size` | int | No | 1 - 1000 | `10` | "redis.pool_size: pool size out of range (expected: 1-1000)" |
| `redis.min_idle_conns` | int | No | 0 - pool_size | `5` | "redis.min_idle_conns: must be 0 to pool_size" |
| `redis.dial_timeout` | duration | No | 1s - 30s | `5s` | "redis.dial_timeout: timeout out of range (expected: 1s-30s)" |
| `redis.read_timeout` | duration | No | 1s - 30s | `3s` | "redis.read_timeout: timeout out of range (expected: 1s-30s)" |
| `redis.write_timeout` | duration | No | 1s - 30s | `3s` | "redis.write_timeout: timeout out of range (expected: 1s-30s)" |
---
### Logging Configuration Validation
| Field | Type | Required | Constraint | Default | Error Message |
|-------|------|----------|------------|---------|---------------|
| `logging.level` | string | No | One of: debug, info, warn, error | `info` | "logging.level: invalid log level (expected: debug, info, warn, error)" |
| `logging.development` | bool | No | true or false | `false` | "logging.development: must be boolean (true/false)" |
#### App Log Validation
| Field | Type | Required | Constraint | Default | Error Message |
|-------|------|----------|------------|---------|---------------|
| `logging.app_log.filename` | string | Yes | Non-empty, valid file path | `logs/app.log` | "logging.app_log.filename: must be non-empty valid file path" |
| `logging.app_log.max_size` | int | No | 1 - 1000 (MB) | `100` | "logging.app_log.max_size: size out of range (expected: 1-1000 MB)" |
| `logging.app_log.max_backups` | int | No | 0 - 999 (0 = keep all) | `30` | "logging.app_log.max_backups: count out of range (expected: 0-999)" |
| `logging.app_log.max_age` | int | No | 1 - 365 (days) | `30` | "logging.app_log.max_age: retention period out of range (expected: 1-365 days)" |
| `logging.app_log.compress` | bool | No | true or false | `true` | "logging.app_log.compress: must be boolean (true/false)" |
#### Access Log Validation
| Field | Type | Required | Constraint | Default | Error Message |
|-------|------|----------|------------|---------|---------------|
| `logging.access_log.filename` | string | Yes | Non-empty, valid file path | `logs/access.log` | "logging.access_log.filename: must be non-empty valid file path" |
| `logging.access_log.max_size` | int | No | 1 - 1000 (MB) | `500` | "logging.access_log.max_size: size out of range (expected: 1-1000 MB)" |
| `logging.access_log.max_backups` | int | No | 0 - 999 (0 = keep all) | `90` | "logging.access_log.max_backups: count out of range (expected: 0-999)" |
| `logging.access_log.max_age` | int | No | 1 - 365 (days) | `90` | "logging.access_log.max_age: retention period out of range (expected: 1-365 days)" |
| `logging.access_log.compress` | bool | No | true or false | `true` | "logging.access_log.compress: must be boolean (true/false)" |
---
### Middleware Configuration Validation
| Field | Type | Required | Constraint | Default | Error Message |
|-------|------|----------|------------|---------|---------------|
| `middleware.enable_auth` | bool | No | true or false | `true` | "middleware.enable_auth: must be boolean (true/false)" |
| `middleware.enable_rate_limiter` | bool | No | true or false | `false` | "middleware.enable_rate_limiter: must be boolean (true/false)" |
#### Rate Limiter Validation
| Field | Type | Required | Constraint | Default | Error Message |
|-------|------|----------|------------|---------|---------------|
| `middleware.rate_limiter.max` | int | No | 1 - 10000 | `30` | "middleware.rate_limiter.max: request limit out of range (expected: 1-10000)" |
| `middleware.rate_limiter.expiration` | duration | No | 1s - 1h | `1m` | "middleware.rate_limiter.expiration: window duration out of range (expected: 1s-1h)" |
| `middleware.rate_limiter.storage` | string | No | "memory" or "redis" | `memory` | "middleware.rate_limiter.storage: invalid storage type (expected: memory or redis)" |
---
### Validation Implementation Guidelines
**Location**: `pkg/config/loader.go` - `Validate()` function
**Validation Order**:
1. **Required fields check** - Fail fast if critical fields missing
2. **Type validation** - Viper handles basic type conversion
3. **Range validation** - Check numeric bounds
4. **Format validation** - Validate string formats (HOST:PORT, file paths)
5. **Cross-field validation** - Check dependencies (e.g., min_idle_conns <= pool_size)
**Example Implementation Pattern**:
```go
func (c *Config) Validate() error {
// Required fields
if c.Server.Address == "" {
return fmt.Errorf("Invalid configuration: server.address: must be non-empty (current value: empty)")
}
// Range validation
if c.Server.ReadTimeout < 5*time.Second || c.Server.ReadTimeout > 300*time.Second {
return fmt.Errorf("Invalid configuration: server.read_timeout: duration out of range (current value: %s, expected: 5s-300s)", c.Server.ReadTimeout)
}
// Format validation
if !strings.Contains(c.Redis.Address, ":") {
return fmt.Errorf("Invalid configuration: redis.address: invalid format (current value: %s, expected: HOST:PORT)", c.Redis.Address)
}
// Cross-field validation
if c.Redis.MinIdleConns > c.Redis.PoolSize {
return fmt.Errorf("Invalid configuration: redis.min_idle_conns: must not exceed pool_size (current value: %d, pool_size: %d)", c.Redis.MinIdleConns, c.Redis.PoolSize)
}
return nil
}
```
**Testing Requirements**:
- Unit tests MUST cover all validation rules (T020 in tasks.md)
- Test valid configurations (should pass)
- Test each constraint violation (should fail with correct error message)
- Test edge cases (min/max boundaries)
- Test malformed YAML (should be caught by Viper)

View File

@@ -0,0 +1,218 @@
# Optional Improvements for 001-fiber-middleware-integration
**Created**: 2025-11-11
**Status**: Deferred - Address after core implementation
**Source**: `/speckit.analyze` findings (Medium/Low priority)
---
## Medium Priority Improvements
### A2: Log Retention Policy Clarity
- **Issue**: FR-006 mentions "configured retention policy" but doesn't specify units/format
- **Current Impact**: Implementation will need to infer format
- **Recommendation**: Add to spec.md FR-006:
```
Retention policy specified in days (integer), e.g., 30 for app logs, 90 for access logs.
Implemented via Lumberjack MaxAge parameter.
```
- **Effort**: 10 minutes (documentation only)
---
### A3: Rate Limiting Default Values
- **Issue**: FR-018a says "configurable requests per time window" but no default values
- **Current Impact**: Developers must guess initial values
- **Recommendation**: Add to spec.md FR-018a:
```
Default: 100 requests per minute per IP
Supported time units: second (s), minute (m), hour (h)
Example config: max=100, window=1m
```
- **Effort**: 15 minutes (spec update + config.yaml example)
---
### A5: Phase 0 Research Status
- **Issue**: plan.md Phase 0 lists "Best Redis client" as unknown but Technical Context already decided
- **Current Impact**: Confusion about whether research is needed
- **Recommendation**: Update plan.md Phase 0:
- Remove "Best Redis client" from unknowns (already decided: go-redis/redis/v8)
- Or clarify: "Validate go-redis/redis/v8 choice (performance benchmarks, connection pool tuning)"
- **Effort**: 5 minutes (documentation clarity)
---
### A6: Config Validation Rules Location
- **Issue**: T009 says "config loading with validation" but validation rules not documented
- **Current Impact**: Developer must design validation rules during implementation
- **Recommendation**: Create section in data-model.md or spec.md:
```markdown
## Configuration Validation Rules
- server.port: 1024-65535 (int)
- server.host: non-empty string
- redis.addr: host:port format
- logging.max_size: 1-1000 (MB)
- logging.max_age: 1-365 (days)
```
- **Effort**: 30 minutes (requires design decisions)
---
### U2: Access Log Format Specification
- **Issue**: FR-011 mentions logging HTTP requests but doesn't specify JSON schema for access.log
- **Current Impact**: Access log structure determined during implementation
- **Recommendation**: Add to spec.md or data-model.md:
```json
{
"timestamp": "2025-11-10T15:30:45Z",
"level": "info",
"request_id": "550e8400-e29b-...",
"method": "GET",
"path": "/api/v1/users",
"status": 200,
"duration_ms": 45,
"ip": "192.168.1.100",
"user_agent": "Mozilla/5.0...",
"user_id": "12345" // if authenticated
}
```
- **Effort**: 20 minutes (design + documentation)
---
### U3: Panic Response Details
- **Issue**: FR-014 says "HTTP 500 with unified error response" but unclear about stack trace handling
- **Current Impact**: Security concern - should stack traces be in production responses?
- **Recommendation**: Clarify in spec.md FR-014:
```
Development/Staging: Include sanitized error message in response.msg, full stack trace in logs only
Production: Generic error message "Internal server error", full details in logs only
Response format: {"code": 1000, "data": null, "msg": "Internal server error"}
```
- **Effort**: 15 minutes (security consideration + spec update)
---
### U4: Rate Limiter Documentation Format
- **Issue**: FR-020 mentions "documentation" but unclear where (quickstart.md? inline comments? separate docs/)
- **Current Impact**: Documentation may be incomplete or scattered
- **Recommendation**: Specify in spec.md FR-020:
```
Documentation location: quickstart.md section "Enabling Rate Limiting"
Must include: Configuration parameters, per-endpoint setup, testing examples
Code example: Uncomment middleware, adjust max/window, test with curl loop
```
- **Effort**: 10 minutes (clarify requirements)
---
### U5: Non-Writable Log Directory Behavior
- **Issue**: Edge case says "fail to start with clear error" but no HTTP status code if started
- **Current Impact**: Unclear behavior if directory becomes non-writable at runtime
- **Recommendation**: Clarify in spec.md Edge Cases:
```
Startup: Fail immediately with exit code 1 and error message before listening on port
Runtime: If directory becomes non-writable, log error to stderr and return 503 on health check
```
- **Effort**: 15 minutes (design decision + spec update)
---
### U6: Performance Test Task Missing
- **Issue**: plan.md mentions "1000+ req/s capacity" but no performance test task in tasks.md
- **Current Impact**: Performance goal not validated
- **Recommendation**: Add task to tasks.md Phase 10:
```
- [ ] T117a [P] Load test with 1000 req/s for 60s, verify P95 < 200ms (use hey or wrk)
```
- **Effort**: 1 hour (implementation + infrastructure setup)
---
### I1: Logger Instances Documentation
- **Issue**: spec.md mentions Zap but tasks reference appLogger/accessLogger instances
- **Current Impact**: Spec doesn't explicitly require two logger instances
- **Recommendation**: Add to spec.md FR-004 or Key Entities:
```
System maintains two independent Zap logger instances:
- appLogger: For application-level logs (business logic, errors, debug)
- accessLogger: For HTTP access logs (request/response details)
Each instance has separate Lumberjack rotation configuration.
```
- **Effort**: 10 minutes (documentation clarity)
---
### I3: Fail-Closed Timing Validation
- **Issue**: spec.md FR-016b says "immediately" but T063 doesn't validate timing (< 100ms)
- **Current Impact**: "Immediate" is subjective, no performance assertion
- **Recommendation**: Update tasks.md T063:
```
- [ ] T063 [P] [US6] Unit test for Redis unavailable (fail closed with < 100ms response time)
```
Or clarify spec.md: "immediately = same request cycle, no retry delays"
- **Effort**: 5 minutes (clarify requirements OR 30 minutes add timing assertion)
---
## Low Priority Improvements
### D1: Response Format Duplication
- **Issue**: FR-007 and US3 both define unified response format
- **Current Impact**: Redundancy, potential inconsistency if one updated
- **Recommendation**: Keep FR-007 as normative, update US3 acceptance criteria:
```
Change: "the response contains `{...}`"
To: "the response follows unified format defined in FR-007"
```
- **Effort**: 5 minutes (reduce duplication)
---
### D2: Code Quality Task Consolidation
- **Issue**: T096-T099 are four separate tasks that could run in one script
- **Current Impact**: Overhead running tasks sequentially
- **Recommendation**: Consider combining to T096-combined:
```
- [ ] T096 [P] Run code quality checks: gofmt -l ., go vet ./..., golangci-lint run, check doc comments
```
Or keep separate for granular progress tracking (current approach is also valid)
- **Effort**: 15 minutes (script creation) OR keep as-is (no change needed)
---
### I4: Task Count Verification
- **Issue**: plan.md says 126 tasks, tasks.md has T001-T126 (now T127 with T012a)
- **Current Impact**: None - counts match after update
- **Recommendation**: No action needed, already addressed in C1 fix
- **Effort**: 0 minutes (already resolved)
---
## Priority Recommendation
**Before Implementation**:
- None (all HIGH priority issues already fixed)
**During Implementation** (if time permits):
- A6: Config validation rules (needed for T009 implementation)
- U2: Access log format (needed for T050 implementation)
- U3: Panic response details (security consideration)
**After Implementation** (polish phase):
- A2, A3: Add default values to config.yaml and documentation
- U4, U5, I1: Documentation improvements
- U6: Performance testing (if infrastructure available)
- D1, D2, I3: Nice-to-have optimizations
---
## Summary
- **Total Optional Improvements**: 14 items
- **Medium Priority**: 9 items (mostly documentation/specification clarity)
- **Low Priority**: 5 items (minor optimizations, already acceptable as-is)
- **Estimated Total Effort**: ~4 hours for all medium priority items
- **Recommended Approach**: Address medium priority items during implementation when context is fresh

View File

@@ -226,13 +226,12 @@ junhong_cmp_fiber/
3. Fiber middleware execution order and error handling patterns
4. Fiber keyauth middleware customization for Redis token validation
5. Fiber limiter middleware configuration options and IP extraction
6. Redis client selection and connection pool configuration for Go
7. UUID v4 generation in Go (standard library vs third-party)
6. Redis connection pool configuration for go-redis/redis/v8 (already decided)
7. UUID v4 generation using github.com/google/uuid (already in go.mod)
8. Graceful shutdown patterns for config watchers and HTTP server
**Unknowns to Resolve**:
- Best Redis client library for Go (go-redis vs redigo)
- Optimal Viper hot reload implementation (polling vs fsnotify)
- Optimal Viper hot reload implementation (polling vs fsnotify - recommend fsnotify)
- How to properly separate Fiber logger middleware output to access.log
- How to inject custom Zap logger into Fiber recover middleware
- Request ID propagation through Fiber context
@@ -295,7 +294,12 @@ This phase will generate `tasks.md` with implementation steps ordered by depende
- Use `github.com/go-redis/redis/v8` for Redis client (widely adopted, good performance)
- Use `github.com/fsnotify/fsnotify` (Viper's native watcher) for hot reload
- Use `github.com/google/uuid` (already in go.mod) for UUID v4 generation
- Separate Zap logger instances for app.log and access.log
- Separate Zap logger instances for app.log and access.log:
- **appLogger**: Initialized with Lumberjack writer pointing to `app.log` with independent rotation settings (max size, max age, max backups, compression). Used for all application-level logging (business logic, errors, middleware events, debug info).
- **accessLogger**: Initialized with separate Lumberjack writer pointing to `access.log` with independent rotation settings. Used exclusively for HTTP access logs (request method, path, status, duration, request ID, IP, user agent, user ID).
- Both loggers use Zap's structured JSON encoder with consistent field naming.
- Logger instances are global variables (package-level) initialized in `pkg/logger/logger.go` via `InitLoggers()` function called from `main.go` at startup.
- Each logger has its own Lumberjack configuration to enable different retention policies (e.g., 30 days for app.log, 90 days for access.log).
- Middleware order: recover → requestid → logger → keyauth → limiter → handler
### Risk Mitigation:

View File

@@ -1,7 +1,7 @@
# Research: Fiber Middleware Integration
**Feature**: 001-fiber-middleware-integration
**Date**: 2025-11-10
**Feature**: 001-fiber-middleware-integration
**Date**: 2025-11-10
**Phase**: 0 - Research & Discovery
## Overview
@@ -26,20 +26,20 @@ This document resolves technical unknowns and establishes best practices for int
viper.WatchConfig()
viper.OnConfigChange(func(e fsnotify.Event) {
log.Info("Config file changed", zap.String("file", e.Name))
// Reload config atomically
newConfig := &Config{}
if err := viper.Unmarshal(newConfig); err != nil {
log.Error("Failed to reload config", zap.Error(err))
return // Keep existing config
}
// Validate new config
if err := newConfig.Validate(); err != nil {
log.Error("Invalid config", zap.Error(err))
return // Keep existing config
}
// Atomic swap using sync/atomic
atomic.StorePointer(&globalConfig, unsafe.Pointer(newConfig))
})
@@ -191,12 +191,12 @@ type TokenValidator struct {
func (v *TokenValidator) Validate(token string) (string, error) {
ctx, cancel := context.WithTimeout(context.Background(), 50*time.Millisecond)
defer cancel()
// Check Redis availability
if err := v.redis.Ping(ctx).Err(); err != nil {
return "", ErrRedisUnavailable // Fail closed
}
// Get user ID from token
userID, err := v.redis.Get(ctx, constants.RedisAuthTokenKey(token)).Result()
if err == redis.Nil {
@@ -205,7 +205,7 @@ func (v *TokenValidator) Validate(token string) (string, error) {
if err != nil {
return "", fmt.Errorf("redis get: %w", err)
}
return userID, nil
}
@@ -217,12 +217,12 @@ func KeyAuth(validator *validator.TokenValidator, logger *zap.Logger) fiber.Hand
userID, err := validator.Validate(key)
if err != nil {
logger.Warn("Token validation failed",
zap.String("request_id", c.Locals("requestid").(string)),
zap.String("request_id", c.Locals(constants.ContextKeyRequestID).(string)),
zap.Error(err),
)
return false, err
}
// Store user ID in context
c.Locals("user_id", userID)
return true, nil
@@ -413,32 +413,32 @@ func main() {
// Create root context with cancellation
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
// Initialize components with context
cfg := config.Load()
go config.Watch(ctx, cfg) // Pass context to watcher
app := setupApp(cfg)
// Graceful shutdown signal handling
quit := make(chan os.Signal, 1)
signal.Notify(quit, os.Interrupt, syscall.SIGTERM)
go func() {
if err := app.Listen(cfg.Server.Address); err != nil {
log.Fatal("Server failed", zap.Error(err))
}
}()
<-quit // Block until signal
log.Info("Shutting down server...")
cancel() // Cancel context (stops config watcher)
if err := app.ShutdownWithTimeout(30 * time.Second); err != nil {
log.Error("Forced shutdown", zap.Error(err))
}
log.Info("Server stopped")
}
```
@@ -455,7 +455,7 @@ func Watch(ctx context.Context, cfg *Config) {
// Reload config logic
}
})
<-ctx.Done() // Block until cancelled
log.Info("Config watcher stopped")
}
@@ -512,22 +512,22 @@ func TestTokenValidator_Validate(t *testing.T) {
},
// More cases...
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
mockRedis := &mock.Redis{}
tt.setupMock(mockRedis)
validator := NewTokenValidator(mockRedis, zap.NewNop())
userID, err := validator.Validate(tt.token)
if err != tt.wantErr {
t.Errorf("got error %v, want %v", err, tt.wantErr)
}
if userID != tt.wantUser {
t.Errorf("got userID %s, want %s", userID, tt.wantUser)
}
mockRedis.AssertExpectations(t)
})
}
@@ -538,7 +538,7 @@ func TestTokenValidator_Validate(t *testing.T) {
```go
func TestMiddlewareChain(t *testing.T) {
// Start testcontainer Redis
redisContainer, err := testcontainers.GenericContainer(ctx,
redisContainer, err := testcontainers.GenericContainer(ctx,
testcontainers.GenericContainerRequest{
ContainerRequest: testcontainers.ContainerRequest{
Image: "redis:7-alpine",
@@ -548,10 +548,10 @@ func TestMiddlewareChain(t *testing.T) {
})
require.NoError(t, err)
defer redisContainer.Terminate(ctx)
// Setup app with middleware
app := setupTestApp(redisContainer)
// Test cases
tests := []struct {
name string
@@ -571,22 +571,22 @@ func TestMiddlewareChain(t *testing.T) {
},
// More cases...
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if tt.setupToken != nil {
tt.setupToken(redisClient)
}
req := httptest.NewRequest("GET", "/api/v1/test", nil)
for k, v := range tt.headers {
req.Header.Set(k, v)
}
resp, err := app.Test(req)
require.NoError(t, err)
assert.Equal(t, tt.expectedStatus, resp.StatusCode)
// Parse response body and check code
var body response.Response
json.NewDecoder(resp.Body).Decode(&body)

View File

@@ -78,10 +78,10 @@ When API consumers (frontend applications, mobile apps, third-party integrations
**Acceptance Scenarios**:
1. **Given** a valid API request, **When** the request succeeds, **Then** the response contains `{"code": 0, "data": {...}, "msg": "success"}`
2. **Given** an invalid API request, **When** validation fails, **Then** the response contains `{"code": [error_code], "data": null, "msg": "[error description]"}`
3. **Given** any API endpoint, **When** processing completes, **Then** the response structure always includes code, data, and msg fields
4. **Given** list/array data is returned, **When** the response is generated, **Then** the data field contains an array instead of an object
1. **Given** a valid API request, **When** the request succeeds, **Then** the response follows the unified format defined in FR-007 with code 0 and appropriate data
2. **Given** an invalid API request, **When** validation fails, **Then** the response follows the unified format defined in FR-007 with appropriate error code and null data
3. **Given** any API endpoint, **When** processing completes, **Then** the response structure always includes all three required fields (code, data, msg) as specified in FR-007
4. **Given** list/array data is returned, **When** the response is generated, **Then** the data field contains an array instead of an object, maintaining the unified format structure
---
@@ -128,10 +128,10 @@ When external clients make API requests, they must provide a valid authenticatio
**Acceptance Scenarios**:
1. **Given** a request to a protected endpoint, **When** the "token" header is missing, **Then** the system returns HTTP 401 with `{"code": 1001, "data": null, "msg": "Missing authentication token"}`
1. **Given** a request to a protected endpoint, **When** the "token" header is missing, **Then** the system returns HTTP 401 with `{"code": 1001, "data": null, "msg": "缺失认证令牌"}`
2. **Given** a request with a token, **When** the token exists as a key in Redis, **Then** the system retrieves the user ID from the value and allows the request to proceed with user context
3. **Given** a request with a token, **When** the token does not exist in Redis (either never created or TTL expired), **Then** the system returns HTTP 401 with `{"code": 1002, "data": null, "msg": "Invalid or expired token"}`
4. **Given** Redis is unavailable, **When** token validation is attempted, **Then** the system immediately fails closed, logs the Redis connection error, and returns HTTP 503 with `{"code": 1004, "data": null, "msg": "Authentication service unavailable"}` without attempting fallback mechanisms
3. **Given** a request with a token, **When** the token does not exist in Redis (either never created or TTL expired), **Then** the system returns HTTP 401 with `{"code": 1002, "data": null, "msg": "令牌无效或已过期"}`
4. **Given** Redis is unavailable, **When** token validation is attempted, **Then** the system immediately fails closed, logs the Redis connection error, and returns HTTP 503 with `{"code": 1004, "data": null, "msg": "认证服务不可用"}` without attempting fallback mechanisms
---
@@ -145,7 +145,7 @@ The system should provide configurable IP-based rate limiting capabilities that
**Acceptance Scenarios**:
1. **Given** rate limiting is configured and enabled for an endpoint with 100 requests per minute per IP, **When** a client IP exceeds the request limit within the time window, **Then** subsequent requests from that IP return HTTP 429 with `{"code": 1003, "data": null, "msg": "Too many requests"}`
1. **Given** rate limiting is configured and enabled for an endpoint with 100 requests per minute per IP, **When** a client IP exceeds the request limit within the time window, **Then** subsequent requests from that IP return HTTP 429 with `{"code": 1003, "data": null, "msg": "请求过于频繁"}`
2. **Given** the rate limit time window expires, **When** new requests arrive from the same client IP, **Then** the request counter resets and requests are allowed again
3. **Given** rate limiting is disabled (default), **When** any number of requests arrive, **Then** all requests are processed without rate limit checks
4. **Given** rate limiting is enabled, **When** requests arrive from different IP addresses, **Then** each IP address has its own independent request counter and limit
@@ -156,7 +156,9 @@ The system should provide configurable IP-based rate limiting capabilities that
- What happens when the configuration file is deleted while the system is running? (System should log error and continue with current configuration)
- What happens when Redis connection is lost during token validation? (System immediately fails closed, returns HTTP 503 with code 1004, logs connection failure, and does not attempt any fallback authentication)
- What happens when log directory is not writable? (System should fail to start with clear error message)
- What happens when log directory is not writable?
- **At startup**: System MUST fail immediately with exit code 1 and clear error message to stderr before listening on any port (e.g., "Fatal: Cannot write to log directory 'logs/': permission denied")
- **At runtime**: If log directory becomes non-writable after successful startup, system MUST log error to stderr, continue serving requests but return HTTP 503 on health check endpoint until log directory becomes writable again
- What happens when a request ID collision occurs? (With UUID v4, collision probability is negligible: ~1 in 2^122; no special handling needed)
- What happens when configuration hot reload occurs during active request processing? (Configuration changes should not affect in-flight requests)
- What happens when log rotation occurs while writing a log entry? (Log rotation should be atomic and not lose log entries)
@@ -167,31 +169,55 @@ The system should provide configurable IP-based rate limiting capabilities that
### Functional Requirements
- **FR-001**: System MUST load configuration from files using Viper configuration library
- **FR-002**: System MUST support hot reload of configuration files, detecting changes within 5 seconds and applying them without service restart
- **FR-003**: System MUST validate configuration values on load and reject invalid configurations with descriptive error messages
- **FR-004**: System MUST write all logs in structured JSON format using Zap logging library
- **FR-002**: System MUST support hot reload of configuration files using fsnotify-based file system event detection (immediate notification on file changes), with configuration changes applied within 5 seconds of file modification and without service restart. The 5-second window includes file event detection, validation, and atomic configuration swap.
- **FR-003**: System MUST validate configuration values on load and reject invalid configurations with descriptive error messages following the format: `"Invalid configuration: {field_path}: {error_reason} (current value: {value}, expected: {constraint})"`. Validation categories include:
- **Type validation**: All fields match expected types (string, int, bool, duration)
- **Range validation**: Numeric values within acceptable ranges (e.g., server.port: 1024-65535, log.max_size: 1-1000 MB)
- **Required fields**: server.host, server.port, redis.addr, logging.app_log_path, logging.access_log_path
- **Format validation**: Durations use Go duration format (e.g., "5m", "30s"), file paths are absolute or relative valid paths
- **Example error**: `"Invalid configuration: server.port: port number out of range (current value: 80, expected: 1024-65535)"`
- **Complete validation rules**: See data-model.md "Configuration Validation Rules" section for comprehensive field-by-field validation constraints
- **FR-004**: System MUST use Zap structured logging for all application logs with log rotation via Lumberjack.v2 and configurable log levels. The system maintains two independent Zap logger instances:
- **appLogger**: For application-level logs (business logic, errors, middleware events, debug info)
- **accessLogger**: For HTTP access logs (request/response details per FR-011)
- Each logger instance has separate Lumberjack rotation configuration for independent file management
- **FR-004a**: System MUST separate application logs (app.log) and HTTP access logs (access.log) into different files with independent configuration
- **FR-005**: System MUST rotate log files automatically using Lumberjack.v2 based on configurable size and age parameters for both application and access logs
- **FR-006**: System MUST retain log files according to configured retention policy and automatically remove expired logs, with separate retention settings for application and access logs
- **FR-007**: All API responses MUST follow the unified format: `{"code": [number], "data": [object/array/null], "msg": [string]}`
- **FR-006**: System MUST retain log files according to configured retention policy and automatically remove expired logs, with separate retention settings for application and access logs. Retention policy is specified in days (integer) and configured via config file (e.g., `logging.app_log_max_age: 30` for 30-day retention of app.log, `logging.access_log_max_age: 90` for 90-day retention of access.log). Implemented via Lumberjack MaxAge parameter.
- **FR-007**: All API responses MUST follow the unified format: `{"code": [number], "data": [object/array/null], "msg": [string]}`. Examples:
- **Success response**: `{"code": 0, "data": {...}, "msg": "success"}`
- **Error response**: `{"code": [error_code], "data": null, "msg": "[error description]"}`
- **List response**: `{"code": 0, "data": [...], "msg": "success"}`
- The response structure always includes all three fields (code, data, msg) regardless of success or failure
- **FR-008**: System MUST assign a unique request ID to every incoming HTTP request using requestid middleware
- **FR-008a**: Request IDs MUST be generated using UUID v4 format for maximum compatibility with distributed tracing systems and log aggregation tools
- **FR-009**: System MUST include the request ID in all log entries associated with that request
- **FR-010**: System MUST include the request ID in HTTP response headers for client-side tracing
- **FR-011**: System MUST log all HTTP requests with method, path, status code, duration, and request ID using logger middleware
- **FR-011**: System MUST log all HTTP requests with method, path, status code, duration, and request ID using logger middleware. Access logs written to access.log MUST use structured JSON format with fields: timestamp (ISO 8601), level, request_id, method, path, status, duration_ms, ip, user_agent, and user_id (if authenticated). See data-model.md "Access Log Entry Format" for complete schema definition.
- **FR-012**: System MUST automatically recover from panics during request processing using recover middleware
- **FR-013**: When a panic is recovered, system MUST log the full stack trace and error details
- **FR-014**: When a panic is recovered, system MUST return HTTP 500 with unified error response format
- **FR-014**: When a panic is recovered, system MUST return HTTP 500 with unified error response format. Response format: `{"code": 1000, "data": null, "msg": "服务器内部错误"}`. The panic error message detail level MUST be configurable via code constant (not config file) to support different deployment environments:
- **Detailed mode** (default for development): Include sanitized panic message in response.msg (e.g., `"服务器内部错误: runtime error: invalid memory address"`)
- **Simple mode** (for production): Return generic message only (`"服务器内部错误"`)
- **Configuration**: Define constant in `pkg/constants/constants.go` as `const PanicResponseDetailLevel = "detailed"` or `"simple"`, easily changeable by developers before deployment
- **Security**: Full stack trace ALWAYS logged to app.log only, NEVER included in HTTP response regardless of mode
- All response messages MUST use Chinese, not English
- **FR-015**: System MUST validate authentication tokens from the "token" request header using keyauth middleware
- **FR-016**: System MUST check token validity by verifying existence in Redis cache using token string as key
- **FR-016a**: System MUST store tokens in Redis as simple key-value pairs with token as key and user ID as value, using Redis TTL for expiration management
- **FR-016b**: When Redis is unavailable during token validation, system MUST fail closed and return HTTP 503 immediately without fallback or caching mechanisms
- **FR-017**: System MUST return HTTP 401 with appropriate error code and message when token is missing or invalid
- **FR-018**: System MUST provide configurable IP-based rate limiting capability using limiter middleware
- **FR-018a**: Rate limiting MUST track request counts per client IP address with configurable limits (requests per time window)
- **FR-018a**: Rate limiting MUST track request counts per client IP address with configurable limits (requests per time window). Default configuration: 30 requests per minute per IP. Supported time units: second (s), minute (m), hour (h). Configuration example in config file: `limiter.max: 30, limiter.window: 1m`
- **FR-018b**: When rate limit is exceeded, system MUST return HTTP 429 with code 1003 and appropriate error message
- **FR-019**: Rate limiting implementation MUST be provided but disabled by default in initial deployment
- **FR-020**: System MUST include documentation on how to configure and enable rate limiting per endpoint with example configurations
- **FR-020**: System MUST include documentation on how to configure and enable rate limiting per endpoint with example configurations. Documentation MUST be created as a separate file `docs/rate-limiting.md` containing:
- **Configuration parameters**: Detailed explanation of `max`, `expiration`, and `storage` settings
- **Per-endpoint setup**: How to enable/disable rate limiting for specific routes or globally
- **Code examples**: Complete examples showing how to uncomment and configure the limiter middleware in `cmd/api/main.go`
- **Testing guide**: Step-by-step instructions with curl commands to test rate limiting behavior
- **Storage options**: Comparison of memory vs Redis storage backends with use cases
- **Common patterns**: Examples for different scenarios (public API, admin endpoints, webhook receivers)
- **FR-021**: System MUST use consistent error codes across all error scenarios with bilingual (Chinese/English) support
- **FR-022**: Configuration MUST support different environments (development, staging, production) with separate config files
@@ -244,7 +270,7 @@ The system should provide configurable IP-based rate limiting capabilities that
### Measurable Outcomes
- **SC-001**: System administrators can modify any configuration value and see it applied within 5 seconds without service restart
- **SC-001**: System administrators can modify any configuration value in the config file and see it applied within 5 seconds (file event detection + validation + atomic swap) without service restart, verified by observing the configuration change take effect (e.g., log level change reflected in subsequent log entries)
- **SC-002**: All API responses follow the unified `{code, data, msg}` structure with 100% consistency across all endpoints
- **SC-003**: Every HTTP request generates a unique UUID v4 request ID that appears in the X-Request-ID response header and all associated log entries
- **SC-004**: System continues processing new requests within 100ms after recovering from a panic, with zero downtime

View File

@@ -20,13 +20,13 @@
**Purpose**: Project initialization and basic structure
- [ ] T001 Create directory structure: pkg/config/, pkg/logger/, pkg/response/, pkg/errors/, pkg/constants/, pkg/validator/, internal/middleware/, configs/, logs/
- [ ] T002 [P] Setup unified error codes and messages in pkg/errors/codes.go
- [ ] T003 [P] Setup custom error types in pkg/errors/errors.go
- [ ] T004 [P] Setup unified response structure in pkg/response/response.go
- [ ] T005 [P] Setup response code constants in pkg/response/codes.go
- [ ] T006 [P] Setup business constants in pkg/constants/constants.go
- [ ] T007 [P] Setup Redis key generation functions in pkg/constants/redis.go
- [X] T001 Create directory structure: pkg/config/, pkg/logger/, pkg/response/, pkg/errors/, pkg/constants/, pkg/validator/, internal/middleware/, configs/, logs/
- [X] T002 [P] Setup unified error codes and messages in pkg/errors/codes.go
- [X] T003 [P] Setup custom error types in pkg/errors/errors.go
- [X] T004 [P] Setup unified response structure in pkg/response/response.go
- [X] T005 [P] Setup response code constants in pkg/response/codes.go
- [X] T006 [P] Setup business constants in pkg/constants/constants.go
- [X] T007 [P] Setup Redis key generation functions in pkg/constants/redis.go
---
@@ -38,22 +38,23 @@
### Configuration Management (US1 Foundation)
- [ ] T008 Create Config structures with Viper mapstructure tags in pkg/config/config.go
- [ ] T009 Implement config loading with validation in pkg/config/loader.go
- [ ] T010 Implement config hot reload with fsnotify in pkg/config/watcher.go
- [ ] T011 Create default configuration file in configs/config.yaml
- [ ] T012 [P] Create environment-specific configs: config.dev.yaml, config.staging.yaml, config.prod.yaml
- [X] T008 Create Config structures with Viper mapstructure tags in pkg/config/config.go
- [X] T009 Implement config loading with validation in pkg/config/loader.go
- [X] T010 Implement config hot reload with fsnotify in pkg/config/watcher.go
- [X] T011 Create default configuration file in configs/config.yaml
- [X] T012 [P] Create environment-specific configs: config.dev.yaml, config.staging.yaml, config.prod.yaml
- [ ] T012a [P] Unit test for environment-specific config loading (test APP_ENV variable loads correct config file) in pkg/config/loader_test.go
### Logging Infrastructure (US2 Foundation)
- [ ] T013 Initialize Zap logger with JSON encoder in pkg/logger/logger.go
- [ ] T014 Setup Lumberjack rotation for app.log in pkg/logger/rotation.go
- [ ] T015 Setup Lumberjack rotation for access.log in pkg/logger/rotation.go
- [ ] T016 Create Fiber logger middleware adapter in pkg/logger/middleware.go
- [X] T013 Initialize Zap logger with JSON encoder in pkg/logger/logger.go
- [X] T014 Setup Lumberjack rotation for app.log in pkg/logger/logger.go (合并到 T013)
- [X] T015 Setup Lumberjack rotation for access.log in pkg/logger/logger.go (合并到 T013)
- [X] T016 Create Fiber logger middleware adapter in pkg/logger/middleware.go
### Redis Connection (US6 Foundation)
- [ ] T017 Setup Redis client with connection pool configuration in pkg/validator/redis.go or pkg/database/redis.go
- [X] T017 Setup Redis client with connection pool configuration in pkg/validator/token.go (使用 go-redis 直接在 main.go 中初始化)
**Checkpoint**: Foundation ready - user story implementation can now begin in parallel
@@ -258,14 +259,14 @@
- [ ] T093 [P] Create example requests (curl commands) in quickstart.md for all scenarios
- [ ] T094 [P] Document middleware execution order in docs/ or README
- [ ] T095 [P] Add troubleshooting section to quickstart.md
- [ ] T095a [P] Create docs/rate-limiting.md with configuration guide, code examples, testing instructions, storage options comparison, and common usage patterns (implements FR-020)
### Code Quality
- [ ] T096 [P] Add Go doc comments to all exported functions and types
- [ ] T097 [P] Run gofmt on all Go files
- [ ] T098 [P] Run go vet and fix all issues
- [ ] T099 [P] Run golangci-lint and fix critical issues
- [ ] T100 Review all Redis key usage, ensure no hardcoded strings (use constants.RedisAuthTokenKey())
- [ ] T097 [P] Run code quality checks (gofmt, go vet, golangci-lint) on all Go files
- [ ] T098 [P] Fix all formatting, linting, and static analysis issues reported by T097
- [ ] T099 Review all Redis key usage, ensure no hardcoded strings (use constants.RedisAuthTokenKey())
- [ ] T101 Review all error handling, ensure explicit returns (no panic abuse)
- [ ] T102 Review naming conventions (UserID not userId, HTTPServer not HttpServer)
- [ ] T103 Check for Java-style anti-patterns (no I-prefix, no Impl-suffix, no getters/setters)
@@ -468,9 +469,9 @@ Then converge for US7 and Polish.
## Task Summary
- **Total Tasks**: 126
- **Total Tasks**: 127 (updated: added T012a for environment config testing, T095a for rate-limiting.md documentation, consolidated code quality checks from 4 tasks into 3 tasks)
- **Setup Phase**: 7 tasks
- **Foundational Phase**: 10 tasks (BLOCKING)
- **Foundational Phase**: 11 tasks (BLOCKING) - includes T012a
- **User Story 1** (Config): 8 tasks (3 tests + 5 implementation)
- **User Story 2** (Logging): 8 tasks (3 tests + 5 implementation)
- **User Story 3** (Response): 9 tasks (3 tests + 6 implementation)
@@ -478,14 +479,14 @@ Then converge for US7 and Polish.
- **User Story 5** (Recovery): 9 tasks (3 tests + 6 implementation)
- **User Story 6** (Auth): 21 tasks (8 tests + 13 implementation)
- **User Story 7** (Rate Limit): 10 tasks (3 tests + 7 implementation)
- **Polish Phase**: 35 tasks (documentation, quality, security)
- **Polish Phase**: 35 tasks (documentation, quality, security) - includes T095a, consolidated code quality checks
**Parallelizable Tasks**: 45 tasks marked [P]
**Parallelizable Tasks**: 47 tasks marked [P] (added T012a, T095a)
**Test Coverage**:
- Unit tests: 23 tasks
- Unit tests: 24 tasks (added T012a)
- Integration tests: 18 tasks
- Total test tasks: 41 (32% of all tasks)
- Total test tasks: 42 (33% of all tasks)
**Constitution Compliance**:
- ✅ All tasks follow Handler → Service → Store → Model pattern