junhong_cmp_fiber/specs/001-fiber-middleware-integration/quickstart.md

# Quick Start Guide: Fiber Middleware Integration

**Feature**: 001-fiber-middleware-integration  
**Date**: 2025-11-10  
**Phase**: 1 - Design & Contracts

## Overview

This guide helps developers set up and test the Fiber middleware integration locally. It covers environment setup, configuration, running the server, and testing middleware functionality.

---

## Prerequisites

### Required Software

- **Go**: 1.25.1 or higher
- **Redis**: 7.x or higher (for authentication)
- **Git**: For version control

### Check Versions

```bash
go version      # Should show go1.25.1 or higher
redis-server --version  # Should show Redis 7.x
```

---

## Environment Setup

### 1. Install Redis (if not already installed)

**macOS (Homebrew)**:
```bash
brew install redis
brew services start redis
```

**Linux (Ubuntu/Debian)**:
```bash
sudo apt-get update
sudo apt-get install redis-server
sudo systemctl start redis
sudo systemctl enable redis
```

**Verify Redis is running**:
```bash
redis-cli ping
# Should return: PONG
```

### 2. Clone Repository and Install Dependencies

```bash
cd /Users/break/csxjProject/junhong_cmp_fiber

# Install Go dependencies
go mod tidy
```

### 3. Create Configuration File

Create `configs/config.yaml` with the following content:

```yaml
server:
  address: ":3000"
  read_timeout: "10s"
  write_timeout: "10s"
  shutdown_timeout: "30s"
  prefork: false

redis:
  address: "localhost:6379"
  password: ""
  db: 0
  pool_size: 10
  min_idle_conns: 5
  dial_timeout: "5s"
  read_timeout: "3s"
  write_timeout: "3s"

logging:
  level: "info"
  development: false
  app_log:
    filename: "logs/app.log"
    max_size: 100      # MB
    max_backups: 30
    max_age: 30        # days
    compress: true
  access_log:
    filename: "logs/access.log"
    max_size: 500      # MB
    max_backups: 90
    max_age: 90        # days
    compress: true

middleware:
  enable_auth: true
  enable_rate_limiter: false  # Disabled by default
  rate_limiter:
    max: 100                  # requests
    expiration: "1m"          # per minute
    storage: "memory"         # or "redis"
```

### 4. Create Logs Directory

```bash
mkdir -p logs
```

---

## Running the Server

### Development Mode

```bash
# Run API server
go run cmd/api/main.go
```

**Expected output**:
```
2025-11-10T15:30:00Z INFO Server starting {"address": ":3000"}
2025-11-10T15:30:00Z INFO Redis connected {"address": "localhost:6379"}
2025-11-10T15:30:00Z INFO Config watcher started
```

### Production Mode

```bash
# Build binary
go build -o bin/api cmd/api/main.go

# Run binary
./bin/api
```

---

## Testing Middleware

### 1. Health Check (No Authentication)

```bash
curl -i http://localhost:3000/health
```

**Expected response**:
```
HTTP/1.1 200 OK
X-Request-ID: 550e8400-e29b-41d4-a716-446655440000
Content-Type: application/json

{
  "code": 0,
  "data": {
    "status": "healthy",
    "timestamp": "2025-11-10T15:30:45Z"
  },
  "msg": "success",
  "timestamp": "2025-11-10T15:30:45Z"
}
```

**Note**: Check the `X-Request-ID` header - this is a UUID v4 generated by the requestid middleware.

### 2. Missing Token (401 Error)

```bash
curl -i http://localhost:3000/api/v1/users
```

**Expected response**:
```
HTTP/1.1 401 Unauthorized
X-Request-ID: 550e8400-e29b-41d4-a716-446655440001
Content-Type: application/json

{
  "code": 1001,
  "data": null,
  "msg": "Missing authentication token",
  "timestamp": "2025-11-10T15:30:46Z"
}
```

### 3. Invalid Token (401 Error)

```bash
curl -i -H "token: invalid-token-123" http://localhost:3000/api/v1/users
```

**Expected response**:
```
HTTP/1.1 401 Unauthorized
X-Request-ID: 550e8400-e29b-41d4-a716-446655440002
Content-Type: application/json

{
  "code": 1002,
  "data": null,
  "msg": "Invalid or expired token",
  "timestamp": "2025-11-10T15:30:47Z"
}
```

### 4. Create Test Token in Redis

```bash
# Add a test token to Redis (expires in 1 hour)
redis-cli SETEX "auth:token:test-token-abc123" 3600 "user-789"
```

**Verify token**:
```bash
redis-cli GET "auth:token:test-token-abc123"
# Should return: "user-789"
```

### 5. Valid Token (200 Success)

```bash
curl -i -H "token: test-token-abc123" http://localhost:3000/api/v1/users
```

**Expected response**:
```
HTTP/1.1 200 OK
X-Request-ID: 550e8400-e29b-41d4-a716-446655440003
Content-Type: application/json

{
  "code": 0,
  "data": [
    {
      "id": "user-123",
      "name": "John Doe",
      "email": "john@example.com"
    }
  ],
  "msg": "success",
  "timestamp": "2025-11-10T15:30:48Z"
}
```

### 6. Test Panic Recovery

Create a test endpoint that panics (for testing recover middleware):

```bash
curl -i -H "token: test-token-abc123" http://localhost:3000/api/v1/test-panic
```

**Expected behavior**:
- Server does NOT crash
- Returns HTTP 500 with error response
- Panic is logged to `logs/app.log` with stack trace
- Subsequent requests continue to work normally

### 7. Verify Logging

**Application logs** (`logs/app.log`):
```bash
tail -f logs/app.log
```

**Expected log entries** (JSON format):
```json
{"timestamp":"2025-11-10T15:30:45Z","level":"info","message":"Server starting","address":":3000"}
{"timestamp":"2025-11-10T15:30:46Z","level":"warn","message":"Token validation failed","request_id":"550e8400-e29b-41d4-a716-446655440001","error":"missing token"}
```

**Access logs** (`logs/access.log`):
```bash
tail -f logs/access.log
```

**Expected log entries** (JSON format):
```json
{"timestamp":"2025-11-10T15:30:46Z","level":"info","method":"GET","path":"/api/v1/users","status":401,"duration_ms":12.345,"request_id":"550e8400-e29b-41d4-a716-446655440001","ip":"127.0.0.1","user_agent":"curl/7.88.1","user_id":""}
{"timestamp":"2025-11-10T15:30:48Z","level":"info","method":"GET","path":"/api/v1/users","status":200,"duration_ms":23.456,"request_id":"550e8400-e29b-41d4-a716-446655440003","ip":"127.0.0.1","user_agent":"curl/7.88.1","user_id":"user-789"}
```

---

## Testing Configuration Hot Reload

### 1. Modify Configuration While Server is Running

Edit `configs/config.yaml` and change the log level:

```yaml
logging:
  level: "debug"  # Changed from "info"
  # ... rest unchanged
```

### 2. Verify Configuration Reloaded

**Check application logs**:
```bash
tail -f logs/app.log
```

**Expected log entry** (within 5 seconds):
```json
{"timestamp":"2025-11-10T15:31:00Z","level":"info","message":"Config file changed","file":"configs/config.yaml"}
{"timestamp":"2025-11-10T15:31:00Z","level":"info","message":"Configuration reloaded successfully"}
```

### 3. Test Invalid Configuration

Edit `configs/config.yaml` with invalid YAML:

```yaml
server:
  address: ":3000"
  invalid syntax here!!!
```

**Expected behavior**:
- Server continues running with previous valid configuration
- Error logged to `logs/app.log`:
  ```json
  {"timestamp":"2025-11-10T15:32:00Z","level":"error","message":"Failed to reload config","error":"yaml: unmarshal error"}
  ```

### 4. Fix Configuration

Restore valid configuration:

```yaml
server:
  address: ":3000"
  read_timeout: "10s"
  # ... rest of valid config
```

**Expected**: Configuration reloads successfully (logged within 5 seconds).

---

## Testing Rate Limiter (Optional)

Rate limiting is **disabled by default**. To enable and test:

### 1. Enable Rate Limiter in Configuration

Edit `configs/config.yaml`:

```yaml
middleware:
  enable_auth: true
  enable_rate_limiter: true  # 设置为 true 启用限流
  rate_limiter:
    max: 5                    # 每个窗口最大请求数（测试用低值）
    expiration: "1m"          # 时间窗口：1分钟
    storage: "memory"         # 存储方式：memory（内存）或 redis（分布式）
```

**Rate Limiter Configuration Options**:

- **`enable_rate_limiter`**: Set to `true` to enable rate limiting (default: `false`)
- **`max`**: Maximum number of requests allowed per time window
  - Development: `1000` requests/minute (relaxed for testing)
  - Production: `100` requests/minute (stricter limits)
  - Testing: `5` requests/minute (for easy testing)
- **`expiration`**: Time window for rate limiting
  - Supported formats: `"30s"` (30 seconds), `"1m"` (1 minute), `"5m"` (5 minutes), `"1h"` (1 hour)
  - Recommended: `"1m"` for most APIs
- **`storage`**: Storage backend for rate limit counters
  - `"memory"`: In-memory storage (single-server deployments)
    - Pros: Fast, no external dependencies
    - Cons: Limits not shared across server instances, reset on server restart
  - `"redis"`: Redis-based storage (multi-server deployments)
    - Pros: Distributed rate limiting, persistent across restarts
    - Cons: Requires Redis connection, slightly higher latency

**Choosing Storage Backend**:

- Use `"memory"` for:
  - Single-server deployments
  - Development/testing environments
  - When rate limit precision is not critical
  
- Use `"redis"` for:
  - Multi-server/load-balanced deployments
  - When you need consistent limits across all servers
  - Production environments with high availability requirements

### 2. Restart Server (or wait for hot reload)

```bash
# Option 1: Restart server
# Ctrl+C to stop
go run cmd/api/main.go

# Option 2: Wait 5 seconds for automatic config reload
# (if server is already running)
```

### 3. Test Rate Limiting

Make multiple requests rapidly:

```bash
# Run 10 requests in quick succession
for i in {1..10}; do
  curl -w "\nRequest $i: %{http_code}\n" \
       -H "token: test-token-abc123" \
       http://localhost:3000/api/v1/users
  sleep 0.1
done
```

**Expected output**:
```
Request 1: 200
Request 2: 200
Request 3: 200
Request 4: 200
Request 5: 200
Request 6: 429  # Rate limit exceeded (请求过于频繁)
Request 7: 429
Request 8: 429
Request 9: 429
Request 10: 429
```

**Rate limit response** (429 Too Many Requests):
```json
{
  "code": 1003,
  "data": null,
  "msg": "请求过于频繁",
  "timestamp": "2025-11-10T15:35:00Z"
}
```

### 4. Test Per-IP Rate Limiting

Rate limiting is applied **per client IP address**. Different IPs have separate rate limits:

```bash
# Simulate requests from different IPs (requires testing infrastructure)
curl -H "X-Forwarded-For: 192.168.1.1" \
     -H "token: test-token-abc123" \
     http://localhost:3000/api/v1/users
# Returns 200 (separate limit from your local IP)
```

### 5. Wait for Window to Reset

Wait for the time window to expire, then try again:

```bash
# Wait for window expiration (1 minute in this example)
sleep 60

# Try again - limit should be reset
curl -H "token: test-token-abc123" http://localhost:3000/api/v1/users
# Should return 200 again
```

### 6. Test Redis-Based Rate Limiting (Distributed)

For distributed rate limiting across multiple servers:

**Edit `configs/config.yaml`**:
```yaml
middleware:
  enable_rate_limiter: true
  rate_limiter:
    max: 100
    expiration: "1m"
    storage: "redis"  # Changed to redis
```

**Check Redis for rate limit keys**:
```bash
# List rate limit keys in Redis
redis-cli KEYS "rate_limit:*"

# Example output:
# 1) "rate_limit:127.0.0.1"
# 2) "rate_limit:192.168.1.1"

# Check remaining count for an IP
redis-cli GET "rate_limit:127.0.0.1"
# Returns: "5" (requests made in current window)

# Check TTL (time until reset)
redis-cli TTL "rate_limit:127.0.0.1"
# Returns: "45" (45 seconds until window resets)
```

### 7. Disable Rate Limiter

Edit `configs/config.yaml`:

```yaml
middleware:
  enable_rate_limiter: false  # 设置为 false 禁用限流
```

Server will reload config automatically within 5 seconds (no restart needed).

### 8. Rate Limiter Behavior Summary

| Scenario | Behavior |
|----------|----------|
| Rate limiter disabled | All requests pass through (no rate limiting) |
| Under limit | Request processed normally (200) |
| Limit exceeded | Request rejected with 429 status code |
| Window expires | Counter resets, requests allowed again |
| Different IPs | Each IP has independent rate limit counter |
| Memory storage + restart | All counters reset on server restart |
| Redis storage + restart | Counters persist across server restarts |
| Redis unavailable | Rate limiting continues with in-memory fallback |

### 9. Recommended Rate Limit Values

**API Type** | **max** | **expiration** | **storage**
-------------|---------|----------------|------------
Public API (strict) | 60 | "1m" | redis
Public API (relaxed) | 1000 | "1m" | redis
Internal API | 5000 | "1m" | memory
Admin API | 10000 | "1m" | memory
Development/Testing | 1000 | "1m" | memory

### 10. Monitoring Rate Limiting

**Check access logs** for rate limit events:
```bash
# Filter 429 responses (rate limited)
grep '"status":429' logs/access.log | jq .

# Example output:
{
  "timestamp": "2025-11-10T15:35:00Z",
  "level": "info",
  "method": "GET",
  "path": "/api/v1/users",
  "status": 429,
  "duration_ms": 0.123,
  "request_id": "550e8400-e29b-41d4-a716-446655440006",
  "ip": "127.0.0.1",
  "user_agent": "curl/7.88.1",
  "user_id": "user-789"
}
```

**Count rate-limited requests**:
```bash
# Count 429 responses in last hour
grep '"status":429' logs/access.log | grep "$(date -u +%Y-%m-%dT%H)" | wc -l
```

---

## Testing Redis Failure (Fail-Closed Behavior)

### 1. Stop Redis

```bash
# macOS
brew services stop redis

# Linux
sudo systemctl stop redis
```

### 2. Test Authentication

```bash
curl -i -H "token: test-token-abc123" http://localhost:3000/api/v1/users
```

**Expected response** (503 Service Unavailable):
```
HTTP/1.1 503 Service Unavailable
X-Request-ID: 550e8400-e29b-41d4-a716-446655440010
Content-Type: application/json

{
  "code": 1004,
  "data": null,
  "msg": "Authentication service unavailable",
  "timestamp": "2025-11-10T15:40:00Z"
}
```

**Check application logs**:
```json
{"timestamp":"2025-11-10T15:40:00Z","level":"error","message":"Redis unavailable","request_id":"550e8400-e29b-41d4-a716-446655440010","error":"dial tcp [::1]:6379: connect: connection refused"}
```

### 3. Restart Redis

```bash
# macOS
brew services start redis

# Linux
sudo systemctl start redis
```

### 4. Verify Recovery

```bash
curl -i -H "token: test-token-abc123" http://localhost:3000/api/v1/users
# Should return 200 again
```

---

## Request ID Tracing

Every request has a unique UUID v4 identifier that appears in:

1. **Response header**: `X-Request-ID`
2. **Access logs**: `request_id` field
3. **Application logs**: `request_id` field (when included)

### Example Request ID Flow

**Request**:
```bash
curl -i -H "token: test-token-abc123" http://localhost:3000/api/v1/users
```

**Response header**:
```
X-Request-ID: 550e8400-e29b-41d4-a716-446655440020
```

**Access log** (`logs/access.log`):
```json
{"timestamp":"2025-11-10T15:45:00Z","level":"info","method":"GET","path":"/api/v1/users","status":200,"duration_ms":15.234,"request_id":"550e8400-e29b-41d4-a716-446655440020","ip":"127.0.0.1","user_agent":"curl/7.88.1","user_id":"user-789"}
```

**Application log** (`logs/app.log`) - if handler logs something:
```json
{"timestamp":"2025-11-10T15:45:00Z","level":"info","message":"Fetching users","request_id":"550e8400-e29b-41d4-a716-446655440020","user_id":"user-789"}
```

### Search Logs by Request ID

```bash
# Search access logs
grep "550e8400-e29b-41d4-a716-446655440020" logs/access.log

# Search application logs
grep "550e8400-e29b-41d4-a716-446655440020" logs/app.log
```

---

## Log Rotation Testing

### Verify Log Rotation Settings

**Application log rotation** (100MB max, 30 day retention):
```bash
ls -lh logs/app.log*
# Should show app.log and rotated files (app.log.1, app.log.2, etc.)
```

**Access log rotation** (500MB max, 90 day retention):
```bash
ls -lh logs/access.log*
# Should show access.log and rotated files
```

### Trigger Log Rotation (Manual Test)

Generate large number of log entries:

```bash
# Generate 1000 requests (will create logs)
for i in {1..1000}; do
  curl -s -H "token: test-token-abc123" http://localhost:3000/api/v1/users > /dev/null
done
```

**Check log file sizes**:
```bash
du -h logs/
```

**Note**: Rotation happens automatically when size limit is reached. Old files are compressed (`.gz`) if compression is enabled.

---

## Troubleshooting

### Problem: Server won't start

**Check**:
1. Port 3000 is not already in use: `lsof -i :3000`
2. Configuration file is valid YAML: `cat configs/config.yaml`
3. Logs directory exists: `ls -ld logs/`

### Problem: Redis connection fails

**Check**:
1. Redis is running: `redis-cli ping`
2. Redis address in config is correct: `localhost:6379`
3. Redis authentication (if password is set)

### Problem: Token validation always fails

**Check**:
1. Token exists in Redis: `redis-cli GET "auth:token:your-token"`
2. Token hasn't expired (check TTL): `redis-cli TTL "auth:token:your-token"`
3. Token key format is correct: `auth:token:{token_string}`

### Problem: Logs not appearing

**Check**:
1. Log directory has write permissions: `ls -ld logs/`
2. Log level in config: `info` or `debug`
3. Logger is properly initialized (check server startup logs)

### Problem: Configuration hot reload not working

**Check**:
1. Configuration file path is correct
2. File system notifications are working (fsnotify)
3. Server logs show config change events
4. New configuration is valid (invalid config is rejected)

---

## Development Workflow

### 1. Code Changes

Make changes to middleware or configuration code.

### 2. Run Tests

```bash
# Run all tests
go test ./...

# Run tests with coverage
go test -cover ./...

# Run specific test
go test -v ./internal/middleware -run TestKeyAuth
```

### 3. Format Code

```bash
# Format all code
go fmt ./...

# Check formatting
gofmt -l .
```

### 4. Static Analysis

```bash
# Run go vet
go vet ./...

# Run golangci-lint (if installed)
golangci-lint run
```

### 5. Build and Test

```bash
# Build
go build -o bin/api cmd/api/main.go

# Test binary
./bin/api
```

---

## Environment-Specific Configurations

### Development (`configs/config.dev.yaml`)

```yaml
logging:
  level: "debug"       # More verbose logging
  development: true    # Pretty-printed logs (non-JSON)

middleware:
  enable_auth: false   # Optional: disable auth for easier testing
```

**Usage**:
```bash
export CONFIG_ENV=dev
go run cmd/api/main.go
```

### Staging (`configs/config.staging.yaml`)

```yaml
server:
  address: ":8080"

redis:
  address: "redis-staging.example.com:6379"
  password: "staging-password"

logging:
  level: "info"

middleware:
  enable_rate_limiter: true
  rate_limiter:
    max: 1000
    expiration: "1m"
```

### Production (`configs/config.prod.yaml`)

```yaml
server:
  address: ":8080"
  prefork: true        # Multi-process mode for performance

redis:
  address: "redis-prod.example.com:6379"
  password: "prod-password"
  pool_size: 50        # Larger pool for production

logging:
  level: "warn"        # Less verbose
  development: false

middleware:
  enable_rate_limiter: true
  rate_limiter:
    max: 5000
    expiration: "1m"
    storage: "redis"   # Distributed rate limiting
```

---

## Next Steps

After verifying the middleware integration works:

1. **Run `/speckit.tasks`**: Generate implementation tasks
2. **Run `/speckit.implement`**: Execute implementation
3. **Write tests**: Unit and integration tests for all middleware
4. **Update documentation**: Add API endpoint examples
5. **Deploy to staging**: Test in staging environment

---

## Quick Reference

### Redis Commands

```bash
# Set token (expires in 1 hour)
redis-cli SETEX "auth:token:TOKEN" 3600 "USER_ID"

# Get token
redis-cli GET "auth:token:TOKEN"

# Check TTL
redis-cli TTL "auth:token:TOKEN"

# Delete token
redis-cli DEL "auth:token:TOKEN"

# List all tokens (careful in production!)
redis-cli KEYS "auth:token:*"
```

### Curl Examples

```bash
# Health check
curl http://localhost:3000/health

# With token
curl -H "token: TOKEN" http://localhost:3000/api/v1/users

# POST request
curl -X POST \
  -H "token: TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"name":"John","email":"john@example.com"}' \
  http://localhost:3000/api/v1/users

# Show response headers
curl -i -H "token: TOKEN" http://localhost:3000/api/v1/users
```

### Log Tailing

```bash
# Tail application logs
tail -f logs/app.log | jq .

# Tail access logs
tail -f logs/access.log | jq .

# Filter by request ID
tail -f logs/app.log | grep "REQUEST_ID"
```

---

**Status**: Ready for implementation

**Next**: Run `/speckit.tasks` to generate implementation task list