做一次小小的备份,等会又删掉了

This commit is contained in:
2025-11-11 10:09:45 +08:00
parent 37c4404293
commit 9600e5b6e0
35 changed files with 4564 additions and 56 deletions

View File

@@ -0,0 +1,678 @@
# Data Model: Fiber Middleware Integration
**Feature**: 001-fiber-middleware-integration
**Date**: 2025-11-10
**Phase**: 1 - Design & Contracts
## Overview
This document defines the data structures, entities, and models used in the middleware integration feature. All structures follow Go idiomatic design principles with simple, flat structures and no Java-style patterns.
---
## 1. Configuration Model
### Config Structure
**File**: `pkg/config/config.go`
```go
// Config represents the application configuration
type Config struct {
Server ServerConfig `mapstructure:"server"`
Redis RedisConfig `mapstructure:"redis"`
Logging LoggingConfig `mapstructure:"logging"`
Middleware MiddlewareConfig `mapstructure:"middleware"`
}
// ServerConfig contains HTTP server settings
type ServerConfig struct {
Address string `mapstructure:"address"` // e.g., ":3000"
ReadTimeout time.Duration `mapstructure:"read_timeout"` // e.g., "10s"
WriteTimeout time.Duration `mapstructure:"write_timeout"` // e.g., "10s"
ShutdownTimeout time.Duration `mapstructure:"shutdown_timeout"` // e.g., "30s"
Prefork bool `mapstructure:"prefork"` // Multi-process mode
}
// RedisConfig contains Redis connection settings
type RedisConfig struct {
Address string `mapstructure:"address"` // e.g., "localhost:6379"
Password string `mapstructure:"password"` // Leave empty if no auth
DB int `mapstructure:"db"` // Database number (0-15)
PoolSize int `mapstructure:"pool_size"` // Max connections
MinIdleConns int `mapstructure:"min_idle_conns"` // Keep-alive connections
DialTimeout time.Duration `mapstructure:"dial_timeout"` // e.g., "5s"
ReadTimeout time.Duration `mapstructure:"read_timeout"` // e.g., "3s"
WriteTimeout time.Duration `mapstructure:"write_timeout"` // e.g., "3s"
}
// LoggingConfig contains logging settings
type LoggingConfig struct {
Level string `mapstructure:"level"` // debug, info, warn, error
Development bool `mapstructure:"development"` // Enable dev mode (pretty print)
AppLog LogRotationConfig `mapstructure:"app_log"` // Application log settings
AccessLog LogRotationConfig `mapstructure:"access_log"` // HTTP access log settings
}
// LogRotationConfig contains log rotation settings for Lumberjack
type LogRotationConfig struct {
Filename string `mapstructure:"filename"` // Log file path
MaxSize int `mapstructure:"max_size"` // Max size in MB before rotation
MaxBackups int `mapstructure:"max_backups"` // Max number of old files to keep
MaxAge int `mapstructure:"max_age"` // Max days to retain old files
Compress bool `mapstructure:"compress"` // Compress rotated files
}
// MiddlewareConfig contains middleware settings
type MiddlewareConfig struct {
EnableAuth bool `mapstructure:"enable_auth"` // Enable keyauth middleware
EnableRateLimiter bool `mapstructure:"enable_rate_limiter"` // Enable limiter (default: false)
RateLimiter RateLimiterConfig `mapstructure:"rate_limiter"` // Rate limiter settings
}
// RateLimiterConfig contains rate limiter settings
type RateLimiterConfig struct {
Max int `mapstructure:"max"` // Max requests per window
Expiration time.Duration `mapstructure:"expiration"` // Time window (e.g., "1m")
Storage string `mapstructure:"storage"` // "memory" or "redis"
}
// Validate validates configuration values
func (c *Config) Validate() error {
if c.Server.Address == "" {
return errors.New("server address cannot be empty")
}
if c.Server.ReadTimeout <= 0 {
return errors.New("server read timeout must be positive")
}
if c.Redis.Address == "" {
return errors.New("redis address cannot be empty")
}
if c.Redis.PoolSize <= 0 {
return errors.New("redis pool size must be positive")
}
if c.Logging.AppLog.Filename == "" {
return errors.New("app log filename cannot be empty")
}
if c.Logging.AccessLog.Filename == "" {
return errors.New("access log filename cannot be empty")
}
if c.Middleware.RateLimiter.Max <= 0 {
c.Middleware.RateLimiter.Max = 100 // Default
}
return nil
}
```
**Example YAML Configuration** (`configs/config.yaml`):
```yaml
server:
address: ":3000"
read_timeout: "10s"
write_timeout: "10s"
shutdown_timeout: "30s"
prefork: false
redis:
address: "localhost:6379"
password: ""
db: 0
pool_size: 10
min_idle_conns: 5
dial_timeout: "5s"
read_timeout: "3s"
write_timeout: "3s"
logging:
level: "info"
development: false
app_log:
filename: "logs/app.log"
max_size: 100 # MB
max_backups: 30
max_age: 30 # days
compress: true
access_log:
filename: "logs/access.log"
max_size: 500 # MB
max_backups: 90
max_age: 90 # days
compress: true
middleware:
enable_auth: true
enable_rate_limiter: false # Disabled by default
rate_limiter:
max: 100 # requests
expiration: "1m" # per minute
storage: "memory" # or "redis"
```
**Validation Rules**:
- All required fields must be non-empty
- Timeouts must be positive durations
- Pool sizes must be positive integers
- Log filenames must be valid paths
- Rate limiter max must be positive (defaults to 100)
---
## 2. Authentication Model
### AuthToken Entity
**Storage**: Redis key-value pair
**Key Format**: `auth:token:{token_string}` (generated via `constants.RedisAuthTokenKey()`)
**Value Format**: Plain string containing user ID
**TTL**: Managed by Redis (set when token is created)
```go
// Token validation doesn't use a struct - just Redis key-value
// Key: "auth:token:abc123def456"
// Value: "user-789"
// TTL: 3600 seconds (1 hour)
```
**Redis Operations**:
```go
// Store token (example - not part of this feature)
rdb.Set(ctx, constants.RedisAuthTokenKey(token), userID, 1*time.Hour)
// Validate token (this feature)
userID, err := rdb.Get(ctx, constants.RedisAuthTokenKey(token)).Result()
if err == redis.Nil {
// Token not found or expired
}
// Delete token (example - logout feature)
rdb.Del(ctx, constants.RedisAuthTokenKey(token))
```
**Key Generation Function** (`pkg/constants/redis.go`):
```go
// RedisAuthTokenKey generates Redis key for authentication tokens
func RedisAuthTokenKey(token string) string {
return fmt.Sprintf("auth:token:%s", token)
}
```
**Validation Rules**:
- Token must exist as Redis key
- Redis must be available (fail closed if not)
- Token value must be non-empty user ID
- TTL is checked automatically by Redis (expired keys return `redis.Nil`)
---
## 3. Request Context Model
### RequestContext Structure
**File**: `pkg/middleware/context.go` (or stored in Fiber's `c.Locals()`)
```go
// Request context is stored in Fiber's Locals, not a struct
// Access via: c.Locals("key")
// Request ID (set by requestid middleware)
requestID := c.Locals("requestid").(string) // UUID v4 string
// User ID (set by keyauth middleware after validation)
userID, ok := c.Locals("user_id").(string)
if !ok {
// Not authenticated
}
// Start time (for duration calculation)
startTime := c.Locals("start_time").(time.Time)
```
**Context Keys** (constants in `pkg/constants/constants.go`):
```go
const (
// Context keys for Fiber Locals
ContextKeyRequestID = "requestid"
ContextKeyUserID = "user_id"
ContextKeyStartTime = "start_time"
)
```
**Lifecycle**:
1. **requestid middleware**: Sets `requestid` in Locals (UUID v4)
2. **logger middleware**: Sets `start_time` in Locals
3. **keyauth middleware**: Sets `user_id` in Locals (after validation)
4. **handler**: Accesses context values from Locals
5. **logger middleware** (after handler): Calculates duration and logs
---
## 4. Log Entry Models
### Application Log Entry
**Format**: JSON
**Output**: `logs/app.log`
**Logger**: Zap (appLogger instance)
```json
{
"timestamp": "2025-11-10T15:30:45.123Z",
"level": "info",
"logger": "service.user",
"caller": "user/service.go:42",
"message": "User created successfully",
"request_id": "550e8400-e29b-41d4-a716-446655440000",
"user_id": "user-789",
"username": "john_doe",
"ip": "192.168.1.100"
}
```
**Fields**:
- `timestamp`: ISO 8601 format (RFC3339)
- `level`: debug, info, warn, error
- `logger`: Logger name (optional, for structured logging)
- `caller`: Source file and line number
- `message`: Log message
- `request_id`: Request correlation ID (if available)
- `user_id`: Authenticated user ID (if available)
- Custom fields: Any additional context-specific fields
**Zap Usage**:
```go
appLogger.Info("User created successfully",
zap.String("request_id", requestID),
zap.String("user_id", userID),
zap.String("username", username),
zap.String("ip", ip),
)
```
### Access Log Entry
**Format**: JSON
**Output**: `logs/access.log`
**Logger**: Zap (accessLogger instance)
```json
{
"timestamp": "2025-11-10T15:30:45.123Z",
"level": "info",
"method": "POST",
"path": "/api/v1/users",
"status": 200,
"duration_ms": 45.234,
"request_id": "550e8400-e29b-41d4-a716-446655440000",
"ip": "192.168.1.100",
"user_agent": "Mozilla/5.0...",
"user_id": "user-789"
}
```
**Fields**:
- `timestamp`: ISO 8601 format (RFC3339)
- `level`: Always "info" for access logs
- `method`: HTTP method (GET, POST, PUT, DELETE)
- `path`: Request path
- `status`: HTTP status code
- `duration_ms`: Request duration in milliseconds
- `request_id`: Request correlation ID
- `ip`: Client IP address
- `user_agent`: User-Agent header
- `user_id`: Authenticated user ID (if available, empty if not)
**Zap Usage**:
```go
accessLogger.Info("",
zap.String("method", c.Method()),
zap.String("path", c.Path()),
zap.Int("status", c.Response().StatusCode()),
zap.Float64("duration_ms", duration.Seconds()*1000),
zap.String("request_id", requestID),
zap.String("ip", c.IP()),
zap.String("user_agent", c.Get("User-Agent")),
zap.String("user_id", userID),
)
```
---
## 5. API Response Model
### Unified Response Structure
**File**: `pkg/response/response.go`
```go
// Response is the unified API response structure
type Response struct {
Code int `json:"code"` // Application error code (0 = success)
Data interface{} `json:"data"` // Response data (object, array, or null)
Message string `json:"msg"` // Human-readable message
Timestamp string `json:"timestamp"` // ISO 8601 timestamp (optional, can be added)
}
// Success returns a successful response
func Success(c *fiber.Ctx, data interface{}) error {
return c.JSON(Response{
Code: 0,
Data: data,
Message: "success",
Timestamp: time.Now().Format(time.RFC3339),
})
}
// Error returns an error response
func Error(c *fiber.Ctx, httpStatus int, code int, message string) error {
return c.Status(httpStatus).JSON(Response{
Code: code,
Data: nil,
Message: message,
Timestamp: time.Now().Format(time.RFC3339),
})
}
// SuccessWithMessage returns a successful response with custom message
func SuccessWithMessage(c *fiber.Ctx, data interface{}, message string) error {
return c.JSON(Response{
Code: 0,
Data: data,
Message: message,
Timestamp: time.Now().Format(time.RFC3339),
})
}
```
**Success Response Example**:
```json
{
"code": 0,
"data": {
"id": "user-123",
"name": "John Doe",
"email": "john@example.com"
},
"msg": "success",
"timestamp": "2025-11-10T15:30:45Z"
}
```
**Error Response Example**:
```json
{
"code": 1002,
"data": null,
"msg": "Invalid or expired token",
"timestamp": "2025-11-10T15:30:45Z"
}
```
**List Response Example**:
```json
{
"code": 0,
"data": [
{"id": "1", "name": "Item 1"},
{"id": "2", "name": "Item 2"}
],
"msg": "success",
"timestamp": "2025-11-10T15:30:45Z"
}
```
---
## 6. Error Model
### Error Code Constants
**File**: `pkg/errors/codes.go`
```go
// Application error codes
const (
CodeSuccess = 0 // Success
CodeInternalError = 1000 // Internal server error
CodeMissingToken = 1001 // Missing authentication token
CodeInvalidToken = 1002 // Invalid or expired token
CodeTooManyRequests = 1003 // Too many requests (rate limited)
CodeAuthServiceUnavailable = 1004 // Authentication service unavailable (Redis down)
)
// Error messages (bilingual)
var errorMessages = map[int]struct {
EN string
ZH string
}{
CodeSuccess: {"Success", "成功"},
CodeInternalError: {"Internal server error", "内部服务器错误"},
CodeMissingToken: {"Missing authentication token", "缺失认证令牌"},
CodeInvalidToken: {"Invalid or expired token", "令牌无效或已过期"},
CodeTooManyRequests: {"Too many requests", "请求过于频繁"},
CodeAuthServiceUnavailable: {"Authentication service unavailable", "认证服务不可用"},
}
// GetMessage returns error message for given code and language
func GetMessage(code int, lang string) string {
msg, ok := errorMessages[code]
if !ok {
return "Unknown error"
}
if lang == "zh" {
return msg.ZH
}
return msg.EN
}
```
### Custom Error Types
**File**: `pkg/errors/errors.go`
```go
import "errors"
// Standard error types for middleware
var (
ErrMissingToken = errors.New("missing authentication token")
ErrInvalidToken = errors.New("invalid or expired token")
ErrRedisUnavailable = errors.New("redis unavailable")
ErrTooManyRequests = errors.New("too many requests")
)
// AppError represents an application error with code
type AppError struct {
Code int // Application error code
Message string // Error message
Err error // Underlying error (optional)
}
func (e *AppError) Error() string {
if e.Err != nil {
return fmt.Sprintf("%s: %v", e.Message, e.Err)
}
return e.Message
}
func (e *AppError) Unwrap() error {
return e.Err
}
// New creates a new AppError
func New(code int, message string) *AppError {
return &AppError{
Code: code,
Message: message,
}
}
// Wrap wraps an existing error with code and message
func Wrap(code int, message string, err error) *AppError {
return &AppError{
Code: code,
Message: message,
Err: err,
}
}
```
---
## 7. Rate Limit State Model
### Rate Limit Tracking
**Storage**: In-memory (default) or Redis (for distributed)
**Managed by**: Fiber limiter middleware (internal storage)
```go
// Rate limit state is managed internally by Fiber limiter
// For memory storage: map[string]*limiterEntry
// For Redis storage: Redis keys with TTL
// Memory storage structure (internal to Fiber)
type limiterEntry struct {
count int // Current request count
expiration time.Time // Window expiration time
}
// Redis storage structure (if using Redis storage)
// Key: "ratelimit:{ip_address}"
// Value: JSON {"count": 5, "expiration": "2025-11-10T15:31:00Z"}
// TTL: Same as expiration window
```
**Key Generation** (for custom Redis storage):
```go
// pkg/constants/redis.go
func RedisRateLimitKey(ip string) string {
return fmt.Sprintf("ratelimit:%s", ip)
}
```
**Access Pattern**:
- Middleware checks rate limit state before handler
- Increment counter on each request
- Reset counter when window expires
- Return 429 when limit exceeded
---
## Entity Relationship Diagram
```
┌─────────────────┐
│ Configuration │ (YAML file)
│ - Server │
│ - Redis │
│ - Logging │
│ - Middleware │
└────────┬────────┘
│ loads into
┌─────────────────┐ ┌──────────────────┐
│ Application │────▶│ Redis Client │
│ (main.go) │ │ (connection │
└────────┬────────┘ │ pool) │
│ └──────────┬───────┘
│ creates │
▼ │ validates tokens
┌─────────────────┐ │
│ Middleware │ │
│ Chain: │ │
│ - Recover │ │
│ - RequestID │ │
│ - Logger │ │
│ - KeyAuth │◀───────────────┘
│ - RateLimiter │
└────────┬────────┘
│ processes
┌─────────────────┐ ┌──────────────────┐
│ Handlers │────▶│ Response │
│ (API │ │ {code, data, │
│ endpoints) │ │ msg} │
└─────────────────┘ └──────────────────┘
│ logs to
┌─────────────────┐
│ Log Files │
│ - app.log │ (Zap + Lumberjack)
│ - access.log │
└─────────────────┘
```
---
## Data Flow Diagram
### Request Processing Flow
```
1. HTTP Request arrives
2. Recover Middleware (catch panics)
3. RequestID Middleware (generate UUID v4)
→ Store in c.Locals("requestid")
4. Logger Middleware (start)
→ Store start_time in c.Locals("start_time")
5. KeyAuth Middleware
→ Extract token from header
→ Call TokenValidator.Validate(token)
→ Validate with Redis: GET auth:token:{token}
→ If valid: Store user_id in c.Locals("user_id")
→ If invalid: Return 401 with error code
→ If Redis down: Return 503 with error code
6. [RateLimiter Middleware] (if enabled)
→ Check rate limit for c.IP()
→ If exceeded: Return 429 with error code
7. Handler (business logic)
→ Access c.Locals("requestid"), c.Locals("user_id")
→ Process request
→ Return response via response.Success() or response.Error()
8. Logger Middleware (end)
→ Calculate duration
→ Log to access.log with all context
9. HTTP Response sent
```
---
## Summary
**Key Entities**:
1. **Config**: Application configuration (YAML → struct)
2. **AuthToken**: Redis key-value (token → user ID)
3. **RequestContext**: Fiber Locals (requestid, user_id, start_time)
4. **LogEntry**: JSON logs (app.log, access.log)
5. **Response**: Unified API response ({code, data, msg})
6. **Error**: Error codes and custom types
7. **RateLimitState**: Managed by Fiber limiter (memory or Redis)
**Design Principles**:
- ✅ Simple, flat structures (no deep nesting)
- ✅ Direct field access (no getters/setters)
- ✅ Composition over inheritance
- ✅ Explicit error handling
- ✅ Go naming conventions (URL, ID, HTTP)
- ✅ No Java-style patterns (no I-prefix, no Impl-suffix)
**Next**: Generate API contracts (OpenAPI specification)