csxj2026/junhong_cmp_fiber

Fork 0

Files

huang 9600e5b6e0 做一次小小的备份,等会又删掉了

2025-11-11 10:09:45 +08:00

16 KiB

Raw Blame History

Quick Start Guide: Fiber Middleware Integration

Feature: 001-fiber-middleware-integration
Date: 2025-11-10
Phase: 1 - Design & Contracts

Overview

This guide helps developers set up and test the Fiber middleware integration locally. It covers environment setup, configuration, running the server, and testing middleware functionality.

Prerequisites

Required Software

Go: 1.25.1 or higher
Redis: 7.x or higher (for authentication)
Git: For version control

Check Versions

go version      # Should show go1.25.1 or higher
redis-server --version  # Should show Redis 7.x

Environment Setup

1. Install Redis (if not already installed)

macOS (Homebrew):

brew install redis
brew services start redis

Linux (Ubuntu/Debian):

sudo apt-get update
sudo apt-get install redis-server
sudo systemctl start redis
sudo systemctl enable redis

Verify Redis is running:

redis-cli ping
# Should return: PONG

2. Clone Repository and Install Dependencies

cd /Users/break/csxjProject/junhong_cmp_fiber

# Install Go dependencies
go mod tidy

3. Create Configuration File

Create configs/config.yaml with the following content:

server:
  address: ":3000"
  read_timeout: "10s"
  write_timeout: "10s"
  shutdown_timeout: "30s"
  prefork: false

redis:
  address: "localhost:6379"
  password: ""
  db: 0
  pool_size: 10
  min_idle_conns: 5
  dial_timeout: "5s"
  read_timeout: "3s"
  write_timeout: "3s"

logging:
  level: "info"
  development: false
  app_log:
    filename: "logs/app.log"
    max_size: 100      # MB
    max_backups: 30
    max_age: 30        # days
    compress: true
  access_log:
    filename: "logs/access.log"
    max_size: 500      # MB
    max_backups: 90
    max_age: 90        # days
    compress: true

middleware:
  enable_auth: true
  enable_rate_limiter: false  # Disabled by default
  rate_limiter:
    max: 100                  # requests
    expiration: "1m"          # per minute
    storage: "memory"         # or "redis"

4. Create Logs Directory

mkdir -p logs

Running the Server

Development Mode

# Run API server
go run cmd/api/main.go

Expected output:

2025-11-10T15:30:00Z INFO Server starting {"address": ":3000"}
2025-11-10T15:30:00Z INFO Redis connected {"address": "localhost:6379"}
2025-11-10T15:30:00Z INFO Config watcher started

Production Mode

# Build binary
go build -o bin/api cmd/api/main.go

# Run binary
./bin/api

Testing Middleware

1. Health Check (No Authentication)

curl -i http://localhost:3000/health

Expected response:

HTTP/1.1 200 OK
X-Request-ID: 550e8400-e29b-41d4-a716-446655440000
Content-Type: application/json

{
  "code": 0,
  "data": {
    "status": "healthy",
    "timestamp": "2025-11-10T15:30:45Z"
  },
  "msg": "success",
  "timestamp": "2025-11-10T15:30:45Z"
}

Note: Check the X-Request-ID header - this is a UUID v4 generated by the requestid middleware.

2. Missing Token (401 Error)

curl -i http://localhost:3000/api/v1/users

Expected response:

HTTP/1.1 401 Unauthorized
X-Request-ID: 550e8400-e29b-41d4-a716-446655440001
Content-Type: application/json

{
  "code": 1001,
  "data": null,
  "msg": "Missing authentication token",
  "timestamp": "2025-11-10T15:30:46Z"
}

3. Invalid Token (401 Error)

curl -i -H "token: invalid-token-123" http://localhost:3000/api/v1/users

Expected response:

HTTP/1.1 401 Unauthorized
X-Request-ID: 550e8400-e29b-41d4-a716-446655440002
Content-Type: application/json

{
  "code": 1002,
  "data": null,
  "msg": "Invalid or expired token",
  "timestamp": "2025-11-10T15:30:47Z"
}

4. Create Test Token in Redis

# Add a test token to Redis (expires in 1 hour)
redis-cli SETEX "auth:token:test-token-abc123" 3600 "user-789"

Verify token:

redis-cli GET "auth:token:test-token-abc123"
# Should return: "user-789"

5. Valid Token (200 Success)

curl -i -H "token: test-token-abc123" http://localhost:3000/api/v1/users

Expected response:

HTTP/1.1 200 OK
X-Request-ID: 550e8400-e29b-41d4-a716-446655440003
Content-Type: application/json

{
  "code": 0,
  "data": [
    {
      "id": "user-123",
      "name": "John Doe",
      "email": "john@example.com"
    }
  ],
  "msg": "success",
  "timestamp": "2025-11-10T15:30:48Z"
}

6. Test Panic Recovery

Create a test endpoint that panics (for testing recover middleware):

curl -i -H "token: test-token-abc123" http://localhost:3000/api/v1/test-panic

Expected behavior:

Server does NOT crash
Returns HTTP 500 with error response
Panic is logged to logs/app.log with stack trace
Subsequent requests continue to work normally

7. Verify Logging

Application logs (logs/app.log):

tail -f logs/app.log

Expected log entries (JSON format):

{"timestamp":"2025-11-10T15:30:45Z","level":"info","message":"Server starting","address":":3000"}
{"timestamp":"2025-11-10T15:30:46Z","level":"warn","message":"Token validation failed","request_id":"550e8400-e29b-41d4-a716-446655440001","error":"missing token"}

Access logs (logs/access.log):

tail -f logs/access.log

Expected log entries (JSON format):

{"timestamp":"2025-11-10T15:30:46Z","level":"info","method":"GET","path":"/api/v1/users","status":401,"duration_ms":12.345,"request_id":"550e8400-e29b-41d4-a716-446655440001","ip":"127.0.0.1","user_agent":"curl/7.88.1","user_id":""}
{"timestamp":"2025-11-10T15:30:48Z","level":"info","method":"GET","path":"/api/v1/users","status":200,"duration_ms":23.456,"request_id":"550e8400-e29b-41d4-a716-446655440003","ip":"127.0.0.1","user_agent":"curl/7.88.1","user_id":"user-789"}

Testing Configuration Hot Reload

1. Modify Configuration While Server is Running

Edit configs/config.yaml and change the log level:

logging:
  level: "debug"  # Changed from "info"
  # ... rest unchanged

2. Verify Configuration Reloaded

Check application logs:

tail -f logs/app.log

Expected log entry (within 5 seconds):

{"timestamp":"2025-11-10T15:31:00Z","level":"info","message":"Config file changed","file":"configs/config.yaml"}
{"timestamp":"2025-11-10T15:31:00Z","level":"info","message":"Configuration reloaded successfully"}

3. Test Invalid Configuration

Edit configs/config.yaml with invalid YAML:

server:
  address: ":3000"
  invalid syntax here!!!

Expected behavior:

Server continues running with previous valid configuration

Error logged to logs/app.log:

{"timestamp":"2025-11-10T15:32:00Z","level":"error","message":"Failed to reload config","error":"yaml: unmarshal error"}

4. Fix Configuration

Restore valid configuration:

server:
  address: ":3000"
  read_timeout: "10s"
  # ... rest of valid config

Expected: Configuration reloads successfully (logged within 5 seconds).

Testing Rate Limiter (Optional)

Rate limiting is disabled by default. To enable and test:

1. Enable Rate Limiter in Configuration

Edit configs/config.yaml:

middleware:
  enable_auth: true
  enable_rate_limiter: true  # Changed to true
  rate_limiter:
    max: 5                    # Low limit for testing
    expiration: "1m"          # 1 minute window
    storage: "memory"

2. Restart Server

# Ctrl+C to stop
go run cmd/api/main.go

3. Test Rate Limiting

Make multiple requests rapidly:

# Run 10 requests in quick succession
for i in {1..10}; do
  curl -w "\nRequest $i: %{http_code}\n" \
       -H "token: test-token-abc123" \
       http://localhost:3000/api/v1/users
  sleep 0.1
done

Expected output:

Request 1: 200
Request 2: 200
Request 3: 200
Request 4: 200
Request 5: 200
Request 6: 429  # Rate limit exceeded
Request 7: 429
Request 8: 429
Request 9: 429
Request 10: 429

Rate limit response (429):

{
  "code": 1003,
  "data": null,
  "msg": "Too many requests",
  "timestamp": "2025-11-10T15:35:00Z"
}

4. Wait for Window to Reset

Wait 1 minute, then try again:

sleep 60
curl -H "token: test-token-abc123" http://localhost:3000/api/v1/users
# Should return 200 again

5. Disable Rate Limiter

Edit configs/config.yaml:

middleware:
  enable_rate_limiter: false  # Back to false

Server will reload config automatically (no restart needed).

Testing Redis Failure (Fail-Closed Behavior)

1. Stop Redis

# macOS
brew services stop redis

# Linux
sudo systemctl stop redis

2. Test Authentication

curl -i -H "token: test-token-abc123" http://localhost:3000/api/v1/users

Expected response (503 Service Unavailable):

HTTP/1.1 503 Service Unavailable
X-Request-ID: 550e8400-e29b-41d4-a716-446655440010
Content-Type: application/json

{
  "code": 1004,
  "data": null,
  "msg": "Authentication service unavailable",
  "timestamp": "2025-11-10T15:40:00Z"
}

Check application logs:

{"timestamp":"2025-11-10T15:40:00Z","level":"error","message":"Redis unavailable","request_id":"550e8400-e29b-41d4-a716-446655440010","error":"dial tcp [::1]:6379: connect: connection refused"}

3. Restart Redis

# macOS
brew services start redis

# Linux
sudo systemctl start redis

4. Verify Recovery

curl -i -H "token: test-token-abc123" http://localhost:3000/api/v1/users
# Should return 200 again

Request ID Tracing

Every request has a unique UUID v4 identifier that appears in:

Response header: X-Request-ID
Access logs: request_id field
Application logs: request_id field (when included)

Example Request ID Flow

Request:

curl -i -H "token: test-token-abc123" http://localhost:3000/api/v1/users

Response header:

X-Request-ID: 550e8400-e29b-41d4-a716-446655440020

Access log (logs/access.log):

{"timestamp":"2025-11-10T15:45:00Z","level":"info","method":"GET","path":"/api/v1/users","status":200,"duration_ms":15.234,"request_id":"550e8400-e29b-41d4-a716-446655440020","ip":"127.0.0.1","user_agent":"curl/7.88.1","user_id":"user-789"}

Application log (logs/app.log) - if handler logs something:

{"timestamp":"2025-11-10T15:45:00Z","level":"info","message":"Fetching users","request_id":"550e8400-e29b-41d4-a716-446655440020","user_id":"user-789"}

Search Logs by Request ID

# Search access logs
grep "550e8400-e29b-41d4-a716-446655440020" logs/access.log

# Search application logs
grep "550e8400-e29b-41d4-a716-446655440020" logs/app.log

Log Rotation Testing

Verify Log Rotation Settings

Application log rotation (100MB max, 30 day retention):

ls -lh logs/app.log*
# Should show app.log and rotated files (app.log.1, app.log.2, etc.)

Access log rotation (500MB max, 90 day retention):

ls -lh logs/access.log*
# Should show access.log and rotated files

Trigger Log Rotation (Manual Test)

Generate large number of log entries:

# Generate 1000 requests (will create logs)
for i in {1..1000}; do
  curl -s -H "token: test-token-abc123" http://localhost:3000/api/v1/users > /dev/null
done

Check log file sizes:

du -h logs/

Note: Rotation happens automatically when size limit is reached. Old files are compressed (.gz) if compression is enabled.

Troubleshooting

Problem: Server won't start

Check:

Port 3000 is not already in use: lsof -i :3000
Configuration file is valid YAML: cat configs/config.yaml
Logs directory exists: ls -ld logs/

Problem: Redis connection fails

Check:

Redis is running: redis-cli ping
Redis address in config is correct: localhost:6379
Redis authentication (if password is set)

Problem: Token validation always fails

Check:

Token exists in Redis: redis-cli GET "auth:token:your-token"
Token hasn't expired (check TTL): redis-cli TTL "auth:token:your-token"
Token key format is correct: auth:token:{token_string}

Problem: Logs not appearing

Check:

Log directory has write permissions: ls -ld logs/
Log level in config: info or debug
Logger is properly initialized (check server startup logs)

Problem: Configuration hot reload not working

Check:

Configuration file path is correct
File system notifications are working (fsnotify)
Server logs show config change events
New configuration is valid (invalid config is rejected)

Development Workflow

1. Code Changes

Make changes to middleware or configuration code.

2. Run Tests

# Run all tests
go test ./...

# Run tests with coverage
go test -cover ./...

# Run specific test
go test -v ./internal/middleware -run TestKeyAuth

3. Format Code

# Format all code
go fmt ./...

# Check formatting
gofmt -l .

4. Static Analysis

# Run go vet
go vet ./...

# Run golangci-lint (if installed)
golangci-lint run

5. Build and Test

# Build
go build -o bin/api cmd/api/main.go

# Test binary
./bin/api

Environment-Specific Configurations

Development (`configs/config.dev.yaml`)

logging:
  level: "debug"       # More verbose logging
  development: true    # Pretty-printed logs (non-JSON)

middleware:
  enable_auth: false   # Optional: disable auth for easier testing

Usage:

export CONFIG_ENV=dev
go run cmd/api/main.go

Staging (`configs/config.staging.yaml`)

server:
  address: ":8080"

redis:
  address: "redis-staging.example.com:6379"
  password: "staging-password"

logging:
  level: "info"

middleware:
  enable_rate_limiter: true
  rate_limiter:
    max: 1000
    expiration: "1m"

Production (`configs/config.prod.yaml`)

server:
  address: ":8080"
  prefork: true        # Multi-process mode for performance

redis:
  address: "redis-prod.example.com:6379"
  password: "prod-password"
  pool_size: 50        # Larger pool for production

logging:
  level: "warn"        # Less verbose
  development: false

middleware:
  enable_rate_limiter: true
  rate_limiter:
    max: 5000
    expiration: "1m"
    storage: "redis"   # Distributed rate limiting

Next Steps

After verifying the middleware integration works:

Run /speckit.tasks: Generate implementation tasks
Run /speckit.implement: Execute implementation
Write tests: Unit and integration tests for all middleware
Update documentation: Add API endpoint examples
Deploy to staging: Test in staging environment

Quick Reference

Redis Commands

# Set token (expires in 1 hour)
redis-cli SETEX "auth:token:TOKEN" 3600 "USER_ID"

# Get token
redis-cli GET "auth:token:TOKEN"

# Check TTL
redis-cli TTL "auth:token:TOKEN"

# Delete token
redis-cli DEL "auth:token:TOKEN"

# List all tokens (careful in production!)
redis-cli KEYS "auth:token:*"

Curl Examples

# Health check
curl http://localhost:3000/health

# With token
curl -H "token: TOKEN" http://localhost:3000/api/v1/users

# POST request
curl -X POST \
  -H "token: TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"name":"John","email":"john@example.com"}' \
  http://localhost:3000/api/v1/users

# Show response headers
curl -i -H "token: TOKEN" http://localhost:3000/api/v1/users

Log Tailing

# Tail application logs
tail -f logs/app.log | jq .

# Tail access logs
tail -f logs/access.log | jq .

# Filter by request ID
tail -f logs/app.log | grep "REQUEST_ID"

Status: Ready for implementation

Next: Run /speckit.tasks to generate implementation task list

16 KiB Raw Blame History

Quick Start Guide: Fiber Middleware Integration

Overview

Prerequisites

Required Software

Check Versions

Environment Setup

1. Install Redis (if not already installed)

2. Clone Repository and Install Dependencies

3. Create Configuration File

4. Create Logs Directory

Running the Server

Development Mode

Production Mode

Testing Middleware

1. Health Check (No Authentication)

2. Missing Token (401 Error)

3. Invalid Token (401 Error)

4. Create Test Token in Redis

5. Valid Token (200 Success)

6. Test Panic Recovery

7. Verify Logging

Testing Configuration Hot Reload

1. Modify Configuration While Server is Running

2. Verify Configuration Reloaded

3. Test Invalid Configuration

4. Fix Configuration

Testing Rate Limiter (Optional)

1. Enable Rate Limiter in Configuration

2. Restart Server

3. Test Rate Limiting

4. Wait for Window to Reset

5. Disable Rate Limiter

Testing Redis Failure (Fail-Closed Behavior)

1. Stop Redis

2. Test Authentication

3. Restart Redis

4. Verify Recovery

Request ID Tracing

Example Request ID Flow

Search Logs by Request ID

Log Rotation Testing

Verify Log Rotation Settings

Trigger Log Rotation (Manual Test)

Troubleshooting

Problem: Server won't start

Problem: Redis connection fails

Problem: Token validation always fails

Problem: Logs not appearing

Problem: Configuration hot reload not working

Development Workflow

1. Code Changes

2. Run Tests

3. Format Code

4. Static Analysis

5. Build and Test

Environment-Specific Configurations

Development (configs/config.dev.yaml)

Staging (configs/config.staging.yaml)

Production (configs/config.prod.yaml)

Next Steps

Quick Reference

Redis Commands

Curl Examples

Log Tailing

16 KiB

Raw Blame History

Development (`configs/config.dev.yaml`)

Staging (`configs/config.staging.yaml`)

Production (`configs/config.prod.yaml`)