junhong_cmp_fiber/specs/001-fiber-middleware-integration/quickstart.md

Quick Start Guide: Fiber Middleware Integration

Feature: 001-fiber-middleware-integration
Date: 2025-11-10
Phase: 1 - Design & Contracts

Overview

This guide helps developers set up and test the Fiber middleware integration locally. It covers environment setup, configuration, running the server, and testing middleware functionality.

---

Prerequisites

Required Software

• Go: 1.25.1 or higher
• Redis: 7.x or higher (for authentication)
• Git: For version control

Check Versions

go version      # Should show go1.25.1 or higher
redis-server --version  # Should show Redis 7.x

---

Environment Setup

1. Install Redis (if not already installed)

macOS (Homebrew):

brew install redis
brew services start redis

Linux (Ubuntu/Debian):

sudo apt-get update
sudo apt-get install redis-server
sudo systemctl start redis
sudo systemctl enable redis

Verify Redis is running:

redis-cli ping
# Should return: PONG

2. Clone Repository and Install Dependencies

cd /Users/break/csxjProject/junhong_cmp_fiber

# Install Go dependencies
go mod tidy

3. Create Configuration File

Create configs/config.yaml with the following content:

server:
  address: ":3000"
  read_timeout: "10s"
  write_timeout: "10s"
  shutdown_timeout: "30s"
  prefork: false

redis:
  address: "localhost:6379"
  password: ""
  db: 0
  pool_size: 10
  min_idle_conns: 5
  dial_timeout: "5s"
  read_timeout: "3s"
  write_timeout: "3s"

logging:
  level: "info"
  development: false
  app_log:
    filename: "logs/app.log"
    max_size: 100      # MB
    max_backups: 30
    max_age: 30        # days
    compress: true
  access_log:
    filename: "logs/access.log"
    max_size: 500      # MB
    max_backups: 90
    max_age: 90        # days
    compress: true

middleware:
  enable_auth: true
  enable_rate_limiter: false  # Disabled by default
  rate_limiter:
    max: 100                  # requests
    expiration: "1m"          # per minute
    storage: "memory"         # or "redis"

4. Create Logs Directory

mkdir -p logs

---

Running the Server

Development Mode

# Run API server
go run cmd/api/main.go

Expected output:

2025-11-10T15:30:00Z INFO Server starting {"address": ":3000"}
2025-11-10T15:30:00Z INFO Redis connected {"address": "localhost:6379"}
2025-11-10T15:30:00Z INFO Config watcher started

Production Mode

# Build binary
go build -o bin/api cmd/api/main.go

# Run binary
./bin/api

---

Testing Middleware

1. Health Check (No Authentication)

curl -i http://localhost:3000/health

Expected response:

HTTP/1.1 200 OK
X-Request-ID: 550e8400-e29b-41d4-a716-446655440000
Content-Type: application/json

{
  "code": 0,
  "data": {
    "status": "healthy",
    "timestamp": "2025-11-10T15:30:45Z"
  },
  "msg": "success",
  "timestamp": "2025-11-10T15:30:45Z"
}

Note: Check the X-Request-ID header - this is a UUID v4 generated by the requestid middleware.

2. Missing Token (401 Error)

curl -i http://localhost:3000/api/v1/users

Expected response:

HTTP/1.1 401 Unauthorized
X-Request-ID: 550e8400-e29b-41d4-a716-446655440001
Content-Type: application/json

{
  "code": 1001,
  "data": null,
  "msg": "Missing authentication token",
  "timestamp": "2025-11-10T15:30:46Z"
}

3. Invalid Token (401 Error)

curl -i -H "token: invalid-token-123" http://localhost:3000/api/v1/users

Expected response:

HTTP/1.1 401 Unauthorized
X-Request-ID: 550e8400-e29b-41d4-a716-446655440002
Content-Type: application/json

{
  "code": 1002,
  "data": null,
  "msg": "Invalid or expired token",
  "timestamp": "2025-11-10T15:30:47Z"
}

4. Create Test Token in Redis

# Add a test token to Redis (expires in 1 hour)
redis-cli SETEX "auth:token:test-token-abc123" 3600 "user-789"

Verify token:

redis-cli GET "auth:token:test-token-abc123"
# Should return: "user-789"

5. Valid Token (200 Success)

curl -i -H "token: test-token-abc123" http://localhost:3000/api/v1/users

Expected response:

HTTP/1.1 200 OK
X-Request-ID: 550e8400-e29b-41d4-a716-446655440003
Content-Type: application/json

{
  "code": 0,
  "data": [
    {
      "id": "user-123",
      "name": "John Doe",
      "email": "john@example.com"
    }
  ],
  "msg": "success",
  "timestamp": "2025-11-10T15:30:48Z"
}

6. Test Panic Recovery

Create a test endpoint that panics (for testing recover middleware):

curl -i -H "token: test-token-abc123" http://localhost:3000/api/v1/test-panic

Expected behavior:

• Server does NOT crash
• Returns HTTP 500 with error response
• Panic is logged to logs/app.log with stack trace
• Subsequent requests continue to work normally

7. Verify Logging

Application logs (logs/app.log):

tail -f logs/app.log

Expected log entries (JSON format):

{"timestamp":"2025-11-10T15:30:45Z","level":"info","message":"Server starting","address":":3000"}
{"timestamp":"2025-11-10T15:30:46Z","level":"warn","message":"Token validation failed","request_id":"550e8400-e29b-41d4-a716-446655440001","error":"missing token"}

Access logs (logs/access.log):

tail -f logs/access.log

Expected log entries (JSON format):

{"timestamp":"2025-11-10T15:30:46Z","level":"info","method":"GET","path":"/api/v1/users","status":401,"duration_ms":12.345,"request_id":"550e8400-e29b-41d4-a716-446655440001","ip":"127.0.0.1","user_agent":"curl/7.88.1","user_id":""}
{"timestamp":"2025-11-10T15:30:48Z","level":"info","method":"GET","path":"/api/v1/users","status":200,"duration_ms":23.456,"request_id":"550e8400-e29b-41d4-a716-446655440003","ip":"127.0.0.1","user_agent":"curl/7.88.1","user_id":"user-789"}

---

Testing Configuration Hot Reload

1. Modify Configuration While Server is Running

Edit configs/config.yaml and change the log level:

logging:
  level: "debug"  # Changed from "info"
  # ... rest unchanged

2. Verify Configuration Reloaded

Check application logs:

tail -f logs/app.log

Expected log entry (within 5 seconds):

{"timestamp":"2025-11-10T15:31:00Z","level":"info","message":"Config file changed","file":"configs/config.yaml"}
{"timestamp":"2025-11-10T15:31:00Z","level":"info","message":"Configuration reloaded successfully"}

3. Test Invalid Configuration

Edit configs/config.yaml with invalid YAML:

server:
  address: ":3000"
  invalid syntax here!!!

Expected behavior:

• Server continues running with previous valid configuration
• Error logged to logs/app.log:

  {"timestamp":"2025-11-10T15:32:00Z","level":"error","message":"Failed to reload config","error":"yaml: unmarshal error"}
  

4. Fix Configuration

Restore valid configuration:

server:
  address: ":3000"
  read_timeout: "10s"
  # ... rest of valid config

Expected: Configuration reloads successfully (logged within 5 seconds).

---

Testing Rate Limiter (Optional)

Rate limiting is disabled by default. To enable and test:

1. Enable Rate Limiter in Configuration

Edit configs/config.yaml:

middleware:
  enable_auth: true
  enable_rate_limiter: true  # 设置为 true 启用限流
  rate_limiter:
    max: 5                    # 每个窗口最大请求数（测试用低值）
    expiration: "1m"          # 时间窗口：1分钟
    storage: "memory"         # 存储方式：memory（内存）或 redis（分布式）

Rate Limiter Configuration Options:

• enable_rate_limiter: Set to true to enable rate limiting (default: false)
• max: Maximum number of requests allowed per time window
  • Development: 1000 requests/minute (relaxed for testing)
  • Production: 100 requests/minute (stricter limits)
  • Testing: 5 requests/minute (for easy testing)
• expiration: Time window for rate limiting
  • Supported formats: "30s" (30 seconds), "1m" (1 minute), "5m" (5 minutes), "1h" (1 hour)
  • Recommended: "1m" for most APIs
• storage: Storage backend for rate limit counters
  • "memory": In-memory storage (single-server deployments)
    • Pros: Fast, no external dependencies
    • Cons: Limits not shared across server instances, reset on server restart
  • "redis": Redis-based storage (multi-server deployments)
    • Pros: Distributed rate limiting, persistent across restarts
    • Cons: Requires Redis connection, slightly higher latency

Choosing Storage Backend:

• Use "memory" for:

  • Single-server deployments
  • Development/testing environments
  • When rate limit precision is not critical

• Use "redis" for:

  • Multi-server/load-balanced deployments
  • When you need consistent limits across all servers
  • Production environments with high availability requirements

2. Restart Server (or wait for hot reload)

# Option 1: Restart server
# Ctrl+C to stop
go run cmd/api/main.go

# Option 2: Wait 5 seconds for automatic config reload
# (if server is already running)

3. Test Rate Limiting

Make multiple requests rapidly:

# Run 10 requests in quick succession
for i in {1..10}; do
  curl -w "\nRequest $i: %{http_code}\n" \
       -H "token: test-token-abc123" \
       http://localhost:3000/api/v1/users
  sleep 0.1
done

Expected output:

Request 1: 200
Request 2: 200
Request 3: 200
Request 4: 200
Request 5: 200
Request 6: 429  # Rate limit exceeded (请求过于频繁)
Request 7: 429
Request 8: 429
Request 9: 429
Request 10: 429

Rate limit response (429 Too Many Requests):

{
  "code": 1003,
  "data": null,
  "msg": "请求过于频繁",
  "timestamp": "2025-11-10T15:35:00Z"
}

4. Test Per-IP Rate Limiting

Rate limiting is applied per client IP address. Different IPs have separate rate limits:

# Simulate requests from different IPs (requires testing infrastructure)
curl -H "X-Forwarded-For: 192.168.1.1" \
     -H "token: test-token-abc123" \
     http://localhost:3000/api/v1/users
# Returns 200 (separate limit from your local IP)

5. Wait for Window to Reset

Wait for the time window to expire, then try again:

# Wait for window expiration (1 minute in this example)
sleep 60

# Try again - limit should be reset
curl -H "token: test-token-abc123" http://localhost:3000/api/v1/users
# Should return 200 again

6. Test Redis-Based Rate Limiting (Distributed)

For distributed rate limiting across multiple servers:

Edit configs/config.yaml:

middleware:
  enable_rate_limiter: true
  rate_limiter:
    max: 100
    expiration: "1m"
    storage: "redis"  # Changed to redis

Check Redis for rate limit keys:

# List rate limit keys in Redis
redis-cli KEYS "rate_limit:*"

# Example output:
# 1) "rate_limit:127.0.0.1"
# 2) "rate_limit:192.168.1.1"

# Check remaining count for an IP
redis-cli GET "rate_limit:127.0.0.1"
# Returns: "5" (requests made in current window)

# Check TTL (time until reset)
redis-cli TTL "rate_limit:127.0.0.1"
# Returns: "45" (45 seconds until window resets)

7. Disable Rate Limiter

Edit configs/config.yaml:

middleware:
  enable_rate_limiter: false  # 设置为 false 禁用限流

Server will reload config automatically within 5 seconds (no restart needed).

8. Rate Limiter Behavior Summary

Scenario	Behavior
Rate limiter disabled	All requests pass through (no rate limiting)
Under limit	Request processed normally (200)
Limit exceeded	Request rejected with 429 status code
Window expires	Counter resets, requests allowed again
Different IPs	Each IP has independent rate limit counter
Memory storage + restart	All counters reset on server restart
Redis storage + restart	Counters persist across server restarts
Redis unavailable	Rate limiting continues with in-memory fallback

9. Recommended Rate Limit Values

API Type	max	expiration	storage
Public API (strict)	60	"1m"	redis
Public API (relaxed)	1000	"1m"	redis
Internal API	5000	"1m"	memory
Admin API	10000	"1m"	memory
Development/Testing	1000	"1m"	memory

10. Monitoring Rate Limiting

Check access logs for rate limit events:

# Filter 429 responses (rate limited)
grep '"status":429' logs/access.log | jq .

# Example output:
{
  "timestamp": "2025-11-10T15:35:00Z",
  "level": "info",
  "method": "GET",
  "path": "/api/v1/users",
  "status": 429,
  "duration_ms": 0.123,
  "request_id": "550e8400-e29b-41d4-a716-446655440006",
  "ip": "127.0.0.1",
  "user_agent": "curl/7.88.1",
  "user_id": "user-789"
}

Count rate-limited requests:

# Count 429 responses in last hour
grep '"status":429' logs/access.log | grep "$(date -u +%Y-%m-%dT%H)" | wc -l

---

Testing Redis Failure (Fail-Closed Behavior)

1. Stop Redis

# macOS
brew services stop redis

# Linux
sudo systemctl stop redis

2. Test Authentication

curl -i -H "token: test-token-abc123" http://localhost:3000/api/v1/users

Expected response (503 Service Unavailable):

HTTP/1.1 503 Service Unavailable
X-Request-ID: 550e8400-e29b-41d4-a716-446655440010
Content-Type: application/json

{
  "code": 1004,
  "data": null,
  "msg": "Authentication service unavailable",
  "timestamp": "2025-11-10T15:40:00Z"
}

Check application logs:

{"timestamp":"2025-11-10T15:40:00Z","level":"error","message":"Redis unavailable","request_id":"550e8400-e29b-41d4-a716-446655440010","error":"dial tcp [::1]:6379: connect: connection refused"}

3. Restart Redis

# macOS
brew services start redis

# Linux
sudo systemctl start redis

4. Verify Recovery

curl -i -H "token: test-token-abc123" http://localhost:3000/api/v1/users
# Should return 200 again

---

Request ID Tracing

Every request has a unique UUID v4 identifier that appears in:

1. Response header: X-Request-ID
2. Access logs: request_id field
3. Application logs: request_id field (when included)

Example Request ID Flow

Request:

curl -i -H "token: test-token-abc123" http://localhost:3000/api/v1/users

Response header:

X-Request-ID: 550e8400-e29b-41d4-a716-446655440020

Access log (logs/access.log):

{"timestamp":"2025-11-10T15:45:00Z","level":"info","method":"GET","path":"/api/v1/users","status":200,"duration_ms":15.234,"request_id":"550e8400-e29b-41d4-a716-446655440020","ip":"127.0.0.1","user_agent":"curl/7.88.1","user_id":"user-789"}

Application log (logs/app.log) - if handler logs something:

{"timestamp":"2025-11-10T15:45:00Z","level":"info","message":"Fetching users","request_id":"550e8400-e29b-41d4-a716-446655440020","user_id":"user-789"}

Search Logs by Request ID

# Search access logs
grep "550e8400-e29b-41d4-a716-446655440020" logs/access.log

# Search application logs
grep "550e8400-e29b-41d4-a716-446655440020" logs/app.log

---

Log Rotation Testing

Verify Log Rotation Settings

Application log rotation (100MB max, 30 day retention):

ls -lh logs/app.log*
# Should show app.log and rotated files (app.log.1, app.log.2, etc.)

Access log rotation (500MB max, 90 day retention):

ls -lh logs/access.log*
# Should show access.log and rotated files

Trigger Log Rotation (Manual Test)

Generate large number of log entries:

# Generate 1000 requests (will create logs)
for i in {1..1000}; do
  curl -s -H "token: test-token-abc123" http://localhost:3000/api/v1/users > /dev/null
done

Check log file sizes:

du -h logs/

Note: Rotation happens automatically when size limit is reached. Old files are compressed (.gz) if compression is enabled.

---

Troubleshooting

Problem: Server won't start

Check:

1. Port 3000 is not already in use: lsof -i :3000
2. Configuration file is valid YAML: cat configs/config.yaml
3. Logs directory exists: ls -ld logs/

Problem: Redis connection fails

Check:

1. Redis is running: redis-cli ping
2. Redis address in config is correct: localhost:6379
3. Redis authentication (if password is set)

Problem: Token validation always fails

Check:

1. Token exists in Redis: redis-cli GET "auth:token:your-token"
2. Token hasn't expired (check TTL): redis-cli TTL "auth:token:your-token"
3. Token key format is correct: auth:token:{token_string}

Problem: Logs not appearing

Check:

1. Log directory has write permissions: ls -ld logs/
2. Log level in config: info or debug
3. Logger is properly initialized (check server startup logs)

Problem: Configuration hot reload not working

Check:

1. Configuration file path is correct
2. File system notifications are working (fsnotify)
3. Server logs show config change events
4. New configuration is valid (invalid config is rejected)

---

Development Workflow

1. Code Changes

Make changes to middleware or configuration code.

2. Run Tests

# Run all tests
go test ./...

# Run tests with coverage
go test -cover ./...

# Run specific test
go test -v ./internal/middleware -run TestKeyAuth

3. Format Code

# Format all code
go fmt ./...

# Check formatting
gofmt -l .

4. Static Analysis

# Run go vet
go vet ./...

# Run golangci-lint (if installed)
golangci-lint run

5. Build and Test

# Build
go build -o bin/api cmd/api/main.go

# Test binary
./bin/api

---

Environment-Specific Configurations

Development (configs/config.dev.yaml)

logging:
  level: "debug"       # More verbose logging
  development: true    # Pretty-printed logs (non-JSON)

middleware:
  enable_auth: false   # Optional: disable auth for easier testing

Usage:

export CONFIG_ENV=dev
go run cmd/api/main.go

Staging (configs/config.staging.yaml)

server:
  address: ":8080"

redis:
  address: "redis-staging.example.com:6379"
  password: "staging-password"

logging:
  level: "info"

middleware:
  enable_rate_limiter: true
  rate_limiter:
    max: 1000
    expiration: "1m"

Production (configs/config.prod.yaml)

server:
  address: ":8080"
  prefork: true        # Multi-process mode for performance

redis:
  address: "redis-prod.example.com:6379"
  password: "prod-password"
  pool_size: 50        # Larger pool for production

logging:
  level: "warn"        # Less verbose
  development: false

middleware:
  enable_rate_limiter: true
  rate_limiter:
    max: 5000
    expiration: "1m"
    storage: "redis"   # Distributed rate limiting

---

Next Steps

After verifying the middleware integration works:

1. Run /speckit.tasks: Generate implementation tasks
2. Run /speckit.implement: Execute implementation
3. Write tests: Unit and integration tests for all middleware
4. Update documentation: Add API endpoint examples
5. Deploy to staging: Test in staging environment

---

Quick Reference

Redis Commands

# Set token (expires in 1 hour)
redis-cli SETEX "auth:token:TOKEN" 3600 "USER_ID"

# Get token
redis-cli GET "auth:token:TOKEN"

# Check TTL
redis-cli TTL "auth:token:TOKEN"

# Delete token
redis-cli DEL "auth:token:TOKEN"

# List all tokens (careful in production!)
redis-cli KEYS "auth:token:*"

Curl Examples

# Health check
curl http://localhost:3000/health

# With token
curl -H "token: TOKEN" http://localhost:3000/api/v1/users

# POST request
curl -X POST \
  -H "token: TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"name":"John","email":"john@example.com"}' \
  http://localhost:3000/api/v1/users

# Show response headers
curl -i -H "token: TOKEN" http://localhost:3000/api/v1/users

Log Tailing

# Tail application logs
tail -f logs/app.log | jq .

# Tail access logs
tail -f logs/access.log | jq .

# Filter by request ID
tail -f logs/app.log | grep "REQUEST_ID"

---

Status: Ready for implementation

Next: Run /speckit.tasks to generate implementation task list