csxj2026/junhong_cmp_fiber

Files

2025-11-11 15:16:38 +08:00

18 KiB

Raw Permalink Blame History

Implementation Plan: Fiber Middleware Integration with Configuration Management

Branch: 001-fiber-middleware-integration | Date: 2025-11-10 | Spec: spec.md Input: Feature specification from /specs/001-fiber-middleware-integration/spec.md

Note: This template is filled in by the /speckit.plan command. See .specify/templates/commands/plan.md for the execution workflow.

Summary

Integrate essential Fiber middleware (logger, recover, requestid, keyauth, limiter) with unified response structure, Viper configuration hot reload, and Zap+Lumberjack logging. This establishes the foundational middleware layer for the 君鸿卡管系统, ensuring proper authentication, logging, error recovery, and request tracing capabilities following Go idiomatic patterns.

Technical Context

Language/Version: Go 1.25.1
Primary Dependencies:

Fiber v2.52.9 (HTTP framework)
Sonic v1.14.2 (JSON encoding/decoding - already integrated)
Viper (configuration with hot reload)
Zap (structured logging)
Lumberjack.v2 (log rotation)
Redis client (for keyauth token validation)

Storage:

Redis (for authentication token validation - token as key, user ID as value with TTL)
Log files (app.log for application logs, access.log for HTTP access logs)

Testing:

Go standard testing framework (testing package)
Table-driven tests for middleware logic
Integration tests for API endpoints with middleware
Mock Redis for keyauth testing

Target Platform: Linux server (production), macOS (development)

Project Type: Single backend web service (Go web application)

Performance Goals:

API response time P95 < 200ms, P99 < 500ms
Middleware overhead < 5ms per request
Configuration hot reload detection within 5 seconds
Log rotation without blocking requests

Constraints:

No external authentication service (Redis-only token validation)
Fail-closed authentication (Redis unavailable = HTTP 503)
Zero downtime for configuration changes (hot reload)
Separate log files with independent retention policies

Scale/Scope:

Foundation for multi-module card management system
~10 initial API endpoints (will grow)
Expected 1000+ req/s capacity
24/7 production availability requirement

Constitution Check

GATE: Must pass before Phase 0 research. Re-check after Phase 1 design.

Tech Stack Adherence:

Feature uses Fiber + Viper + Zap + Lumberjack.v2 + sonic JSON + Redis
No native calls bypass framework (no net/http direct use)
All HTTP operations use Fiber framework
All async tasks use Asynq (N/A for this feature - no async tasks)
Uses Go official toolchain: go fmt, go vet, golangci-lint
Uses Go Modules for dependency management (already initialized)

Code Quality Standards:

Follows Handler → Service → Store → Model architecture (keyauth validation follows this)
Handler layer only handles HTTP, no business logic
Service layer contains business logic with cross-module support
Store layer manages all data access with transaction support
Uses dependency injection via struct fields (not constructor patterns)
Unified error codes in pkg/errors/ (will define auth error codes: 1001-1004)
Unified API responses via pkg/response/ (core requirement: {code, data, msg})
All constants defined in pkg/constants/ (Redis keys, error messages)
All Redis keys managed via key generation functions (no hardcoded strings)
All exported functions/types have Go-style doc comments
Code formatted with gofmt
Follows Effective Go and Go Code Review Comments

Go Idiomatic Design:

Package structure is flat (max 2-3 levels), organized by feature
- Structure: pkg/config/, pkg/logger/, pkg/response/, pkg/errors/, pkg/constants/
- Middleware: internal/middleware/ (flat, no deep nesting)
Interfaces are small (1-3 methods), defined at use site
- TokenValidator interface (2 methods: ValidateToken, IsAvailable)
No Java-style patterns: no I-prefix, no Impl-suffix, no getters/setters
Error handling is explicit (return errors, no panic/recover abuse)
- Recover middleware captures panics only, normal errors use error returns
Uses composition over inheritance
Uses goroutines and channels for config hot reload watcher
Uses context.Context for cancellation and timeouts
Naming follows Go conventions: short receivers, consistent abbreviations (URL, ID, HTTP)
No Hungarian notation or type prefixes
Simple constructors (New/NewXxx), no Builder pattern unless necessary

Testing Standards:

Unit tests for all core business logic (token validation, config loading)
Integration tests for all API endpoints with middleware chain
Tests use Go standard testing framework
Test files named *_test.go in same directory
Test functions use Test prefix, benchmarks use Benchmark prefix
Table-driven tests for multiple test cases (especially middleware scenarios)
Test helpers marked with t.Helper()
Tests are independent (mock Redis, no external dependencies)
Target coverage: 70%+ overall, 90%+ for core business (auth, config)

User Experience Consistency:

All APIs use unified JSON response format: {code, data, msg}
Error responses include clear error codes and bilingual messages
- 1001: Missing authentication token (缺失认证令牌)
- 1002: Invalid or expired token (令牌无效或已过期)
- 1003: Too many requests (请求过于频繁)
- 1004: Authentication service unavailable (认证服务不可用)
RESTful design principles followed
Unified pagination parameters (N/A for this feature)
Time fields use ISO 8601 format (RFC3339) - in log timestamps
Currency amounts use integers (N/A for this feature)

Performance Requirements:

API response time (P95) < 200ms, (P99) < 500ms
- Middleware overhead budgeted at < 5ms per request
Batch operations use bulk queries/inserts (N/A for this feature)
All database queries have appropriate indexes (N/A - Redis only)
List queries implement pagination (N/A for this feature)
Non-realtime operations use async tasks (N/A - all operations are realtime)
Database and Redis connection pools properly configured
- Redis: PoolSize=10, MinIdleConns=5
Uses goroutines/channels for concurrency (config watcher)
Uses context.Context for timeout control (Redis operations)
Uses sync.Pool for frequently allocated objects (if needed for performance)

Project Structure

Documentation (this feature)

specs/001-fiber-middleware-integration/
├── plan.md              # This file (/speckit.plan command output)
├── research.md          # Phase 0 output (/speckit.plan command)
├── data-model.md        # Phase 1 output (/speckit.plan command)
├── quickstart.md        # Phase 1 output (/speckit.plan command)
├── contracts/           # Phase 1 output (/speckit.plan command)
│   └── api.yaml        # OpenAPI spec for unified response format
└── tasks.md             # Phase 2 output (/speckit.tasks command - NOT created by /speckit.plan)

Source Code (repository root)

junhong_cmp_fiber/
├── cmd/
│   ├── api/
│   │   └── main.go              # HTTP server entry point (updated)
│   └── worker/
│       └── main.go              # Worker process (no changes for this feature)
│
├── internal/
│   └── middleware/              # Fiber middleware implementations
│       ├── auth.go              # keyauth middleware integration
│       ├── ratelimit.go         # limiter middleware integration (commented)
│       └── recover.go           # Custom recover middleware with Zap logging
│
├── pkg/
│   ├── config/
│   │   ├── config.go            # Viper configuration management
│   │   ├── loader.go            # Config loading and validation
│   │   └── watcher.go           # Hot reload implementation
│   │
│   ├── logger/
│   │   ├── logger.go            # Zap logger initialization
│   │   ├── rotation.go          # Lumberjack integration
│   │   └── middleware.go        # Fiber logger middleware adapter
│   │
│   ├── response/
│   │   ├── response.go          # Unified response structure and helpers
│   │   └── codes.go             # Response code constants
│   │
│   ├── errors/
│   │   ├── errors.go            # Custom error types
│   │   └── codes.go             # Error code constants (1001-1004)
│   │
│   ├── constants/
│   │   ├── constants.go         # Business constants
│   │   └── redis.go             # Redis key generation functions
│   │
│   └── validator/
│       └── token.go             # Token validation service (Redis interaction)
│
├── configs/
│   ├── config.yaml              # Default configuration
│   ├── config.dev.yaml          # Development environment
│   ├── config.staging.yaml      # Staging environment
│   └── config.prod.yaml         # Production environment
│
├── logs/                        # Log output directory (gitignored)
│   ├── app.log                  # Application logs
│   └── access.log               # HTTP access logs
│
└── tests/
    ├── integration/
    │   ├── middleware_test.go   # Integration tests for middleware chain
    │   └── auth_test.go         # Auth flow integration tests
    └── unit/
        ├── config_test.go       # Config loading and validation tests
        ├── logger_test.go       # Logger tests
        ├── response_test.go     # Response format tests
        └── validator_test.go    # Token validation tests

Structure Decision: Single project structure (Option 1) as this is a backend-only Go web service. The project follows Go's standard layout with cmd/ for entry points, internal/ for private application code, and pkg/ for reusable packages. Configuration files are centralized in configs/ with environment-specific overrides. This structure aligns with Go best practices and the constitution's flat package organization principle.

Complexity Tracking

Fill ONLY if Constitution Check has violations that must be justified

No violations detected. All requirements align with constitution principles.

Phase 0: Research & Discovery

Status: Starting research phase

Research Tasks:

Viper configuration hot reload best practices and implementation patterns
Zap + Lumberjack integration patterns for dual log files (app.log, access.log)
Fiber middleware execution order and error handling patterns
Fiber keyauth middleware customization for Redis token validation
Fiber limiter middleware configuration options and IP extraction
Redis connection pool configuration for go-redis/redis/v8 (already decided)
UUID v4 generation using github.com/google/uuid (already in go.mod)
Graceful shutdown patterns for config watchers and HTTP server

Unknowns to Resolve:

Optimal Viper hot reload implementation (polling vs fsnotify - recommend fsnotify)
How to properly separate Fiber logger middleware output to access.log
How to inject custom Zap logger into Fiber recover middleware
Request ID propagation through Fiber context
Testing strategies for middleware integration tests

Output: research.md with decisions and rationale

Phase 1: Design & Contracts

Prerequisites: Phase 0 research complete

Design Artifacts:

data-model.md: Entity definitions
- Configuration structure (server, redis, logging, middleware settings)
- AuthToken entity (Redis key-value structure)
- Request Context structure (request ID, user ID, metadata)
- Log Entry structure (JSON fields for app.log and access.log)
- Rate Limit State (IP-based tracking structure)
contracts/api.yaml: OpenAPI specification
- Unified response format schema
- Error response schemas (1001-1004)
- Common headers (X-Request-ID, token)
- Example endpoints demonstrating middleware integration
quickstart.md: Developer setup guide
- Environment setup (Go, Redis)
- Configuration file setup (config.yaml)
- Running the server
- Testing middleware (curl examples)
- Enabling/disabling rate limiter

Output: data-model.md, contracts/api.yaml, quickstart.md

Phase 2: Task Generation

Status: To be completed by /speckit.tasks command (NOT part of /speckit.plan)

This phase will generate tasks.md with implementation steps ordered by dependencies.

Notes & Decisions

Key Design Decisions (from spec clarifications):

Log Separation: Application logs (app.log) and HTTP access logs (access.log) in separate files with independent rotation and retention policies
Token Storage: Simple Redis key-value (token → user ID) with Redis TTL for expiration
Auth Failure Mode: Fail closed when Redis unavailable (HTTP 503, no fallback)
Rate Limiting: Per-IP address with configurable requests/time window, disabled by default
Request ID Format: UUID v4 for distributed tracing compatibility
Config Hot Reload: 5-second detection window, atomic updates, invalid config = keep previous

Technical Choices (to be validated in Phase 0):

Use github.com/go-redis/redis/v8 for Redis client (widely adopted, good performance)
Use github.com/fsnotify/fsnotify (Viper's native watcher) for hot reload
Use github.com/google/uuid (already in go.mod) for UUID v4 generation
Separate Zap logger instances for app.log and access.log:
- appLogger: Initialized with Lumberjack writer pointing to app.log with independent rotation settings (max size, max age, max backups, compression). Used for all application-level logging (business logic, errors, middleware events, debug info).
- accessLogger: Initialized with separate Lumberjack writer pointing to access.log with independent rotation settings. Used exclusively for HTTP access logs (request method, path, status, duration, request ID, IP, user agent, user ID).
- Both loggers use Zap's structured JSON encoder with consistent field naming.
- Logger instances are global variables (package-level) initialized in pkg/logger/logger.go via InitLoggers() function called from main.go at startup.
- Each logger has its own Lumberjack configuration to enable different retention policies (e.g., 30 days for app.log, 90 days for access.log).
Middleware order: recover → requestid → logger → keyauth → limiter → handler

Risk Mitigation:

Risk: Configuration hot reload causes in-flight request issues
- Mitigation: Use atomic pointer swap for config updates, don't affect in-flight requests
Risk: Log rotation blocks request processing
- Mitigation: Lumberjack handles rotation atomically, Zap is non-blocking
Risk: Redis connection pool exhaustion under load
- Mitigation: Proper pool sizing (10 connections), timeout configuration, circuit breaker consideration
Risk: Rate limiter memory leak with many unique IPs
- Mitigation: Use Fiber's built-in limiter with storage backend (memory or Redis), TTL cleanup

Phase 1 Constitution Re-Check

Status: ✅ PASSED - All design artifacts comply with constitution

Verification Results:

Go Idiomatic Design ✅
- ✅ Flat package structure: pkg/config/, pkg/logger/, pkg/response/, pkg/errors/
- ✅ Small interfaces: TokenValidator (2 methods only)
- ✅ No Java patterns: No IService, no Impl suffix, no getters/setters
- ✅ Simple structs with direct field access (Config, Response, LogEntry)
- ✅ Explicit error handling with custom error types
- ✅ Composition: Config composed of sub-configs, no inheritance
Tech Stack Compliance ✅
- ✅ Fiber for all HTTP operations (middleware, routing)
- ✅ Viper for configuration with hot reload
- ✅ Zap + Lumberjack for logging
- ✅ go-redis/redis/v8 for Redis client
- ✅ google/uuid for request ID generation
- ✅ No net/http, database/sql, or encoding/json direct usage
Architecture Alignment ✅
- ✅ Clear separation: Middleware → Handler → Service (TokenValidator) → Store (Redis)
- ✅ Dependency injection via struct fields
- ✅ Unified response format in pkg/response/
- ✅ Error codes centralized in pkg/errors/
- ✅ Constants and Redis key functions in pkg/constants/
Performance Requirements ✅
- ✅ Middleware overhead budgeted at <5ms per request
- ✅ Redis connection pool configured (10 connections, 5 idle)
- ✅ Goroutines for config watcher (non-blocking)
- ✅ Context timeouts for Redis operations (50ms)
- ✅ Lumberjack non-blocking log rotation
Testing Strategy ✅
- ✅ Table-driven tests planned (Go idiomatic)
- ✅ Mock Redis for unit tests
- ✅ Integration tests with testcontainers
- ✅ Target coverage defined (70%+ overall, 90%+ core)

Design Quality Metrics:

Zero Java-style anti-patterns detected
All error handling explicit (no panic/recover abuse)
Configuration validation implemented
Graceful shutdown pattern defined
Security: Fail-closed auth, explicit timeout handling

Conclusion: Design is ready for implementation. All artifacts (research.md, data-model.md, contracts/api.yaml, quickstart.md) align with constitution principles. No violations or exceptions required.

Next Steps

✅ Plan created and constitution check passed
✅ Execute Phase 0 research (research.md)
✅ Execute Phase 1 design (data-model.md, contracts/, quickstart.md)
✅ Update agent context with new technologies
✅ Phase 1 Constitution re-check passed
⏳ Run /speckit.tasks to generate implementation tasks
⏳ Run /speckit.implement to execute tasks

Command: /speckit.plan complete. All design artifacts generated and validated.

Ready for: /speckit.tasks command to generate actionable implementation tasks.

18 KiB Raw Permalink Blame History