junhong_cmp_fiber/specs/001-fiber-middleware-integration/data-model.md

Data Model: Fiber Middleware Integration

Feature: 001-fiber-middleware-integration
Date: 2025-11-10
Phase: 1 - Design & Contracts

Overview

This document defines the data structures, entities, and models used in the middleware integration feature. All structures follow Go idiomatic design principles with simple, flat structures and no Java-style patterns.

---

1. Configuration Model

Config Structure

File: pkg/config/config.go

// Config represents the application configuration
type Config struct {
    Server     ServerConfig     `mapstructure:"server"`
    Redis      RedisConfig      `mapstructure:"redis"`
    Logging    LoggingConfig    `mapstructure:"logging"`
    Middleware MiddlewareConfig `mapstructure:"middleware"`
}

// ServerConfig contains HTTP server settings
type ServerConfig struct {
    Address         string        `mapstructure:"address"`           // e.g., ":3000"
    ReadTimeout     time.Duration `mapstructure:"read_timeout"`      // e.g., "10s"
    WriteTimeout    time.Duration `mapstructure:"write_timeout"`     // e.g., "10s"
    ShutdownTimeout time.Duration `mapstructure:"shutdown_timeout"`  // e.g., "30s"
    Prefork         bool          `mapstructure:"prefork"`           // Multi-process mode
}

// RedisConfig contains Redis connection settings
type RedisConfig struct {
    Address      string        `mapstructure:"address"`       // e.g., "localhost:6379"
    Password     string        `mapstructure:"password"`      // Leave empty if no auth
    DB           int           `mapstructure:"db"`            // Database number (0-15)
    PoolSize     int           `mapstructure:"pool_size"`     // Max connections
    MinIdleConns int           `mapstructure:"min_idle_conns"` // Keep-alive connections
    DialTimeout  time.Duration `mapstructure:"dial_timeout"`  // e.g., "5s"
    ReadTimeout  time.Duration `mapstructure:"read_timeout"`  // e.g., "3s"
    WriteTimeout time.Duration `mapstructure:"write_timeout"` // e.g., "3s"
}

// LoggingConfig contains logging settings
type LoggingConfig struct {
    Level       string           `mapstructure:"level"`        // debug, info, warn, error
    Development bool             `mapstructure:"development"`  // Enable dev mode (pretty print)
    AppLog      LogRotationConfig `mapstructure:"app_log"`     // Application log settings
    AccessLog   LogRotationConfig `mapstructure:"access_log"`  // HTTP access log settings
}

// LogRotationConfig contains log rotation settings for Lumberjack
type LogRotationConfig struct {
    Filename   string `mapstructure:"filename"`    // Log file path
    MaxSize    int    `mapstructure:"max_size"`    // Max size in MB before rotation
    MaxBackups int    `mapstructure:"max_backups"` // Max number of old files to keep
    MaxAge     int    `mapstructure:"max_age"`     // Max days to retain old files
    Compress   bool   `mapstructure:"compress"`    // Compress rotated files
}

// MiddlewareConfig contains middleware settings
type MiddlewareConfig struct {
    EnableAuth       bool              `mapstructure:"enable_auth"`        // Enable keyauth middleware
    EnableRateLimiter bool             `mapstructure:"enable_rate_limiter"` // Enable limiter (default: false)
    RateLimiter      RateLimiterConfig `mapstructure:"rate_limiter"`       // Rate limiter settings
}

// RateLimiterConfig contains rate limiter settings
type RateLimiterConfig struct {
    Max        int           `mapstructure:"max"`        // Max requests per window
    Expiration time.Duration `mapstructure:"expiration"` // Time window (e.g., "1m")
    Storage    string        `mapstructure:"storage"`    // "memory" or "redis"
}

// Validate validates configuration values
func (c *Config) Validate() error {
    if c.Server.Address == "" {
        return errors.New("server address cannot be empty")
    }
    if c.Server.ReadTimeout <= 0 {
        return errors.New("server read timeout must be positive")
    }
    if c.Redis.Address == "" {
        return errors.New("redis address cannot be empty")
    }
    if c.Redis.PoolSize <= 0 {
        return errors.New("redis pool size must be positive")
    }
    if c.Logging.AppLog.Filename == "" {
        return errors.New("app log filename cannot be empty")
    }
    if c.Logging.AccessLog.Filename == "" {
        return errors.New("access log filename cannot be empty")
    }
    if c.Middleware.RateLimiter.Max <= 0 {
        c.Middleware.RateLimiter.Max = 100 // Default
    }
    return nil
}

Example YAML Configuration (configs/config.yaml):

server:
  address: ":3000"
  read_timeout: "10s"
  write_timeout: "10s"
  shutdown_timeout: "30s"
  prefork: false

redis:
  address: "localhost:6379"
  password: ""
  db: 0
  pool_size: 10
  min_idle_conns: 5
  dial_timeout: "5s"
  read_timeout: "3s"
  write_timeout: "3s"

logging:
  level: "info"
  development: false
  app_log:
    filename: "logs/app.log"
    max_size: 100      # MB
    max_backups: 30
    max_age: 30        # days
    compress: true
  access_log:
    filename: "logs/access.log"
    max_size: 500      # MB
    max_backups: 90
    max_age: 90        # days
    compress: true

middleware:
  enable_auth: true
  enable_rate_limiter: false  # Disabled by default
  rate_limiter:
    max: 100                  # requests
    expiration: "1m"          # per minute
    storage: "memory"         # or "redis"

Validation Rules:

• All required fields must be non-empty
• Timeouts must be positive durations
• Pool sizes must be positive integers
• Log filenames must be valid paths
• Rate limiter max must be positive (defaults to 100)

---

2. Authentication Model

AuthToken Entity

Storage: Redis key-value pair
Key Format: auth:token:{token_string} (generated via constants.RedisAuthTokenKey())
Value Format: Plain string containing user ID
TTL: Managed by Redis (set when token is created)

// Token validation doesn't use a struct - just Redis key-value
// Key:   "auth:token:abc123def456"
// Value: "user-789"
// TTL:   3600 seconds (1 hour)

Redis Operations:

// Store token (example - not part of this feature)
rdb.Set(ctx, constants.RedisAuthTokenKey(token), userID, 1*time.Hour)

// Validate token (this feature)
userID, err := rdb.Get(ctx, constants.RedisAuthTokenKey(token)).Result()
if err == redis.Nil {
    // Token not found or expired
}

// Delete token (example - logout feature)
rdb.Del(ctx, constants.RedisAuthTokenKey(token))

Key Generation Function (pkg/constants/redis.go):

// RedisAuthTokenKey generates Redis key for authentication tokens
func RedisAuthTokenKey(token string) string {
    return fmt.Sprintf("auth:token:%s", token)
}

Validation Rules:

• Token must exist as Redis key
• Redis must be available (fail closed if not)
• Token value must be non-empty user ID
• TTL is checked automatically by Redis (expired keys return redis.Nil)

---

3. Request Context Model

RequestContext Structure

File: pkg/middleware/context.go (or stored in Fiber's c.Locals())

// Request context is stored in Fiber's Locals, not a struct
// Access via: c.Locals("key")

// Request ID (set by requestid middleware)
requestID := c.Locals("requestid").(string)  // UUID v4 string

// User ID (set by keyauth middleware after validation)
userID, ok := c.Locals("user_id").(string)
if !ok {
    // Not authenticated
}

// Start time (for duration calculation)
startTime := c.Locals("start_time").(time.Time)

Context Keys (constants in pkg/constants/constants.go):

const (
    // Context keys for Fiber Locals
    ContextKeyRequestID = "requestid"
    ContextKeyUserID    = "user_id"
    ContextKeyStartTime = "start_time"
)

Lifecycle:

1. requestid middleware: Sets requestid in Locals (UUID v4)
2. logger middleware: Sets start_time in Locals
3. keyauth middleware: Sets user_id in Locals (after validation)
4. handler: Accesses context values from Locals
5. logger middleware (after handler): Calculates duration and logs

---

4. Log Entry Models

Application Log Entry

Format: JSON
Output: logs/app.log
Logger: Zap (appLogger instance)

{
  "timestamp": "2025-11-10T15:30:45.123Z",
  "level": "info",
  "logger": "service.user",
  "caller": "user/service.go:42",
  "message": "User created successfully",
  "request_id": "550e8400-e29b-41d4-a716-446655440000",
  "user_id": "user-789",
  "username": "john_doe",
  "ip": "192.168.1.100"
}

Fields:

• timestamp: ISO 8601 format (RFC3339)
• level: debug, info, warn, error
• logger: Logger name (optional, for structured logging)
• caller: Source file and line number
• message: Log message
• request_id: Request correlation ID (if available)
• user_id: Authenticated user ID (if available)
• Custom fields: Any additional context-specific fields

Zap Usage:

appLogger.Info("User created successfully",
    zap.String("request_id", requestID),
    zap.String("user_id", userID),
    zap.String("username", username),
    zap.String("ip", ip),
)

Access Log Entry

Format: JSON
Output: logs/access.log
Logger: Zap (accessLogger instance)

{
  "timestamp": "2025-11-10T15:30:45.123Z",
  "level": "info",
  "method": "POST",
  "path": "/api/v1/users",
  "status": 200,
  "duration_ms": 45.234,
  "request_id": "550e8400-e29b-41d4-a716-446655440000",
  "ip": "192.168.1.100",
  "user_agent": "Mozilla/5.0...",
  "user_id": "user-789"
}

Fields:

• timestamp: ISO 8601 format (RFC3339)
• level: Always "info" for access logs
• method: HTTP method (GET, POST, PUT, DELETE)
• path: Request path
• status: HTTP status code
• duration_ms: Request duration in milliseconds
• request_id: Request correlation ID
• ip: Client IP address
• user_agent: User-Agent header
• user_id: Authenticated user ID (if available, empty if not)

Zap Usage:

accessLogger.Info("",
    zap.String("method", c.Method()),
    zap.String("path", c.Path()),
    zap.Int("status", c.Response().StatusCode()),
    zap.Float64("duration_ms", duration.Seconds()*1000),
    zap.String("request_id", requestID),
    zap.String("ip", c.IP()),
    zap.String("user_agent", c.Get("User-Agent")),
    zap.String("user_id", userID),
)

---

5. API Response Model

Unified Response Structure

File: pkg/response/response.go

// Response is the unified API response structure
type Response struct {
    Code      int         `json:"code"`      // Application error code (0 = success)
    Data      interface{} `json:"data"`      // Response data (object, array, or null)
    Message   string      `json:"msg"`       // Human-readable message
    Timestamp string      `json:"timestamp"` // ISO 8601 timestamp (optional, can be added)
}

// Success returns a successful response
func Success(c *fiber.Ctx, data interface{}) error {
    return c.JSON(Response{
        Code:      0,
        Data:      data,
        Message:   "success",
        Timestamp: time.Now().Format(time.RFC3339),
    })
}

// Error returns an error response
func Error(c *fiber.Ctx, httpStatus int, code int, message string) error {
    return c.Status(httpStatus).JSON(Response{
        Code:      code,
        Data:      nil,
        Message:   message,
        Timestamp: time.Now().Format(time.RFC3339),
    })
}

// SuccessWithMessage returns a successful response with custom message
func SuccessWithMessage(c *fiber.Ctx, data interface{}, message string) error {
    return c.JSON(Response{
        Code:      0,
        Data:      data,
        Message:   message,
        Timestamp: time.Now().Format(time.RFC3339),
    })
}

Success Response Example:

{
  "code": 0,
  "data": {
    "id": "user-123",
    "name": "John Doe",
    "email": "john@example.com"
  },
  "msg": "success",
  "timestamp": "2025-11-10T15:30:45Z"
}

Error Response Example:

{
  "code": 1002,
  "data": null,
  "msg": "Invalid or expired token",
  "timestamp": "2025-11-10T15:30:45Z"
}

List Response Example:

{
  "code": 0,
  "data": [
    {"id": "1", "name": "Item 1"},
    {"id": "2", "name": "Item 2"}
  ],
  "msg": "success",
  "timestamp": "2025-11-10T15:30:45Z"
}

---

6. Error Model

Error Code Constants

File: pkg/errors/codes.go

// Application error codes
const (
    CodeSuccess                  = 0    // Success
    CodeInternalError            = 1000 // Internal server error
    CodeMissingToken             = 1001 // Missing authentication token
    CodeInvalidToken             = 1002 // Invalid or expired token
    CodeTooManyRequests          = 1003 // Too many requests (rate limited)
    CodeAuthServiceUnavailable   = 1004 // Authentication service unavailable (Redis down)
)

// Error messages (bilingual)
var errorMessages = map[int]struct {
    EN string
    ZH string
}{
    CodeSuccess:                {"Success", "成功"},
    CodeInternalError:          {"Internal server error", "内部服务器错误"},
    CodeMissingToken:           {"Missing authentication token", "缺失认证令牌"},
    CodeInvalidToken:           {"Invalid or expired token", "令牌无效或已过期"},
    CodeTooManyRequests:        {"Too many requests", "请求过于频繁"},
    CodeAuthServiceUnavailable: {"Authentication service unavailable", "认证服务不可用"},
}

// GetMessage returns error message for given code and language
func GetMessage(code int, lang string) string {
    msg, ok := errorMessages[code]
    if !ok {
        return "Unknown error"
    }
    if lang == "zh" {
        return msg.ZH
    }
    return msg.EN
}

Custom Error Types

File: pkg/errors/errors.go

import "errors"

// Standard error types for middleware
var (
    ErrMissingToken      = errors.New("missing authentication token")
    ErrInvalidToken      = errors.New("invalid or expired token")
    ErrRedisUnavailable  = errors.New("redis unavailable")
    ErrTooManyRequests   = errors.New("too many requests")
)

// AppError represents an application error with code
type AppError struct {
    Code    int    // Application error code
    Message string // Error message
    Err     error  // Underlying error (optional)
}

func (e *AppError) Error() string {
    if e.Err != nil {
        return fmt.Sprintf("%s: %v", e.Message, e.Err)
    }
    return e.Message
}

func (e *AppError) Unwrap() error {
    return e.Err
}

// New creates a new AppError
func New(code int, message string) *AppError {
    return &AppError{
        Code:    code,
        Message: message,
    }
}

// Wrap wraps an existing error with code and message
func Wrap(code int, message string, err error) *AppError {
    return &AppError{
        Code:    code,
        Message: message,
        Err:     err,
    }
}

---

7. Rate Limit State Model

Rate Limit Tracking

Storage: In-memory (default) or Redis (for distributed)
Managed by: Fiber limiter middleware (internal storage)

// Rate limit state is managed internally by Fiber limiter
// For memory storage: map[string]*limiterEntry
// For Redis storage: Redis keys with TTL

// Memory storage structure (internal to Fiber)
type limiterEntry struct {
    count      int       // Current request count
    expiration time.Time // Window expiration time
}

// Redis storage structure (if using Redis storage)
// Key:   "ratelimit:{ip_address}"
// Value: JSON {"count": 5, "expiration": "2025-11-10T15:31:00Z"}
// TTL:   Same as expiration window

Key Generation (for custom Redis storage):

// pkg/constants/redis.go
func RedisRateLimitKey(ip string) string {
    return fmt.Sprintf("ratelimit:%s", ip)
}

Access Pattern:

• Middleware checks rate limit state before handler
• Increment counter on each request
• Reset counter when window expires
• Return 429 when limit exceeded

---

Entity Relationship Diagram

┌─────────────────┐
│  Configuration  │ (YAML file)
│  - Server       │
│  - Redis        │
│  - Logging      │
│  - Middleware   │
└────────┬────────┘
         │ loads into
         ▼
┌─────────────────┐     ┌──────────────────┐
│   Application   │────▶│  Redis Client    │
│   (main.go)     │     │  (connection     │
└────────┬────────┘     │   pool)          │
         │              └──────────┬───────┘
         │ creates                 │
         ▼                         │ validates tokens
┌─────────────────┐                │
│ Middleware      │                │
│ Chain:          │                │
│ - Recover       │                │
│ - RequestID     │                │
│ - Logger        │                │
│ - KeyAuth       │◀───────────────┘
│ - RateLimiter   │
└────────┬────────┘
         │ processes
         ▼
┌─────────────────┐     ┌──────────────────┐
│   Handlers      │────▶│  Response        │
│   (API          │     │  {code, data,    │
│    endpoints)   │     │   msg}           │
└─────────────────┘     └──────────────────┘
         │
         │ logs to
         ▼
┌─────────────────┐
│  Log Files      │
│  - app.log      │ (Zap + Lumberjack)
│  - access.log   │
└─────────────────┘

---

Data Flow Diagram

Request Processing Flow

1. HTTP Request arrives
   ↓
2. Recover Middleware (catch panics)
   ↓
3. RequestID Middleware (generate UUID v4)
   → Store in c.Locals("requestid")
   ↓
4. Logger Middleware (start)
   → Store start_time in c.Locals("start_time")
   ↓
5. KeyAuth Middleware
   → Extract token from header
   → Call TokenValidator.Validate(token)
   → Validate with Redis: GET auth:token:{token}
   → If valid: Store user_id in c.Locals("user_id")
   → If invalid: Return 401 with error code
   → If Redis down: Return 503 with error code
   ↓
6. [RateLimiter Middleware] (if enabled)
   → Check rate limit for c.IP()
   → If exceeded: Return 429 with error code
   ↓
7. Handler (business logic)
   → Access c.Locals("requestid"), c.Locals("user_id")
   → Process request
   → Return response via response.Success() or response.Error()
   ↓
8. Logger Middleware (end)
   → Calculate duration
   → Log to access.log with all context
   ↓
9. HTTP Response sent

---

Summary

Key Entities:

1. Config: Application configuration (YAML → struct)
2. AuthToken: Redis key-value (token → user ID)
3. RequestContext: Fiber Locals (requestid, user_id, start_time)
4. LogEntry: JSON logs (app.log, access.log)
5. Response: Unified API response ({code, data, msg})
6. Error: Error codes and custom types
7. RateLimitState: Managed by Fiber limiter (memory or Redis)

Design Principles:

• ✅ Simple, flat structures (no deep nesting)
• ✅ Direct field access (no getters/setters)
• ✅ Composition over inheritance
• ✅ Explicit error handling
• ✅ Go naming conventions (URL, ID, HTTP)
• ✅ No Java-style patterns (no I-prefix, no Impl-suffix)

Next: Generate API contracts (OpenAPI specification)